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DATA  FUSION  ANALYSIS  FOR  RANGE  TEST 
VALIDATION  SYSTEM 


EXECUTIVE  SUMMARY 

A  range  test  validation  system  (RTVS)  is  being  developed  under  the  sponsorship  of  the 
Defense  Threat  Reduction  Agency.  The  purpose  of  RTVS  is  to  provide  accurate 
identification  and  tracking  of  clouds  of  chemical  agent  simulants  and  a  means  for 
validating  point  and  standoff  chemical  sensors  on  a  test  grid.  As  part  of  the  RTVS 
program,  the  Chemical  Sensing/Chemometrics  Section  (Code  6181)  of  the  Naval 
Research  Laboratory  has  been  tasked  to  develop  the  data  analysis  and  data  fusion 
methods. 

Data  for  chemical  agent  simulants  were  collected  from  various  sensor  systems  during  test 
chemical  releases  at  Dugway  Proving  Ground  in  Utah  in  September  of  2007  and  June  of 
2008.  Members  of  Code  6181  were  present  for  most  of  the  Dugway  testing.  The 
chemical  clouds  released  varied  in  direction,  depending  on  the  wind,  and  in  size, 
depending  on  the  amount  of  chemical  released.  Standoff  and  point  detection  systems 
were  positioned  to  monitor  a  test  grid.  These  included  the  following  RTVS  detection 
systems:  twenty-five  IMS-based  point  sensors  (LCD)  and  Airmar  weather  stations,  and 
five  standoff  IR  imaging  spectrometers  (AIRIS).  The  legacy  Dugway  sensor  systems 
were  comprised  of  several  standoff  IR  cameras  (ORION),  standoff  FTIR  sensors 
(MESH),  photo  ionization-based  point  sensors  (ppbRAE),  and  PWIDS  weather  stations. 

Data  from  these  tests  was  used  to  complete  an  analysis  of  RTVS  data  fusion.  The 
majority  of  the  results  contained  herein  were  presented  to  the  DTRA  program  sponsor  at 
monthly  progress  meetings.  The  conclusions  of  NRL"s  data  fusion  analysis  for  RTVS  are 
as  follows: 

First,  environmental  factors  significantly  affected  the  testing  at  Dugway  Proving 
Grounds.  Winds  induced  significant  variance  in  the  RTVS  data.  Testing  at  night  when 
winds  were  less  variable  did  mitigate  wind  effects,  but  an  inverted  temperature  gradient 
with  respect  to  height  may  have  affected  the  calibration  and  sensing  capabilities  of  the 
standoff  sensors.  Testing  at  night  increased  the  effect,  as  the  temperature  gradient  was 
positive  and  less  steep  during  the  day.  In  addition,  a  large  variation  in  the  RTVS  data 
was  observed  with  respect  to  the  dissemination  method.  Explosive  dissemination 
generated  an  uneven,  globular  release  of  simulant  with  unpredictable  vaporization. 
Dissemination  by  stack  release  produced  a  more  even  distribution  of  vaporized  simulant, 
but  was  not  consistent  with  the  expected  chemical  threat  scenario. 

With  respect  to  the  sensor  systems,  the  effective  calibration  and  baselines  of  the  LCD 
point  sensors  make  them  a  better  choice  than  the  Dugway  ppbRAE  point  sensors  for 
inclusion  in  RTVS.  The  three-dimensional  AIRIS  data  cube  with  individual  voxels 
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covering  the  test  grid  to  a  great  height  is  an  extremely  flexible  format  for  data  fusion. 
The  problems  observed  with  the  AIRIS  tomographic  reconstruction  should  decrease  with 
further  computational  power.  Overall,  the  AIRIS  sensors  provided  cloud-tracking 
capabilities  that  were  similar  to  the  Dugway  MESH  FTIR  sensors,  but  were  at  three-times 
better  resolution.  Cloud  tracking  with  either  AIRIS  or  MESH  was  a  vast  improvement 
over  tracking  with  the  ORION  IR  cameras,  which  typically  lost  track  of  the  cloud  within 
two  minutes  of  its  dissemination.  Although  not  yet  fully  implemented,  real-time 
situational  awareness  is  viable  from  fused  LCD  and  AIRIS  data,  but  the  rate  as  of  the 
2008  test  series  was  limited  to  at  best  one  update  per  minute  (i.e.,  one  data  cube)  with 
coverage  of  the  entire  test  grid. 

For  RTVS  data  fusion,  the  coverage  areas  of  the  LCD  and  AIRIS  sensors  are  effectively 
disparate  and  do  not  overlap,  which  limits  the  perfonnance  gains  possible  with 
multisensor  data  fusion.  However,  the  LCD  and  AIRIS  sensors  jointly  provide 
situational  awareness  in  which  detections  of  simulants  are  clear  indications  of  a  tangible 
chemical  threat,  and  the  absence  of  detections  is  an  accurate  all-clear  signal  for  the  test 
grid  as  a  whole.  The  combined  sensors  are  not  able  to  effectively  distinguish  local 
regions  of  threat  from  regions  of  all  clear  on  the  test  grid  when  a  simulant  has  been 
released  due  to  the  large  distance  between  the  point  sensors,  the  lack  of  overlap  between 
the  point  and  standoff  sensors,  and  the  variances  in  the  data  induced  by  environmental 
conditions,  dissemination  methods,  and  tomographic  reconstruction.  The  presence  of 
significant  numbers  of  false  negatives  means  that  a  Dempster-Shafer  approach  is  the  most 
effective  strategy  for  RTVS  data  fusion.  Dempster-Shafer  treats  all  types  of  infonnation 
as  adding  to  the  total  situational  awareness,  but  remains  flexible  when  the  absence  of 
information  is  uncertain  or  not  infonnative  for  situational  awareness.  Additionally,  a 
number  of  strategies  for  improving  the  quality  of  the  LCD  and  AIRIS  data  are  presented. 
Heuristic  methods  would  be  effective  for  implementing  these  strategies.  Heuristic  rules 
are  also  likely  to  be  a  useful  strategy  for  RTVS  data  fusion. 
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1.  INTRODUCTION 


Under  the  sponsorship  of  the  Defense  Threat  Reduction  Agency  (DTRA),  a  range  test 
validation  system  (RTVS)  is  being  developed  to  track  and  detect  chemical  agent  simulant 
clouds  on  a  test  grid.  The  goal  of  the  program  is  to  determine  in  near  real-time  the  size, 
location,  and  concentration  distribution  of  chemical  clouds.  The  RTVS  will  be  used  to 
validate  point  and  standoff  sensors.  The  data  generated  will  help  develop  advanced 
warning  and  quantitative  analysis  methods  for  fugitive  emissions. 

The  tracking  system  for  RTVS  includes  five  infrared  imaging  spectroradiometers 
(AIRIS)  and  25  point  sensors  (LCD)  based  on  ion  mobility  spectrometry  (IMS).  The  data 
from  these  detection  systems  and  the  weather  information  recorded  at  the  test  sites  are 
used  to  map  the  chemical  clouds  onto  a  common  operational  picture.  The  goal  is  to 
develop  chemical  fate  and  transport  models  that  can  be  used  to  assess  the  perfonnance  of 
detection  systems.  The  fate  and  transport  models  for  the  chemical  cloud  rely  on  the 
fusion  of  image  data,  discrete  sensor  data  at  specific  locations,  and  auxiliary  infonnation 
about  the  test  site  and  weather  conditions.  These  different  types  of  data  need  to  be  fused 
to  provide  a  common  operational  picture  that  describes  the  chemical  clouds  temporally 
and  spatially. 

The  Chemical  Sensing/Chemometrics  Section  (Code  6181)  of  the  Naval  Research 
Laboratory  has  much  experience  with  diverse  sensor  fusion.  Under  ONR  sponsorship, 
NRL  developed  a  multi-sensory  data  fusion  detection  system  to  detect  shipboard  damage 
control  events  such  as  fire,  flooding,  and  pipe  ruptures.  The  detection  system  combined 
regular  and  near-infrared  video  camera,  discrete  spectral  sensors  and  a  microphone  with 
machine  vision,  pattern  recognition,  and  data  fusion  algorithms.  The  detection  system 
detects  in  real  time  flaming  and  smoldering  fires,  pipe  ruptures,  gas  leaks  and  flooding 
events  and  has  a  high  immunity  to  false  positives.  The  Naval  Research  Laboratory  also 
developed  a  data  fusion  framework  for  wide-area  assessment  of  unexploded  ordnance 
(UXO)  using  diverse  data  detection  methods  and  intelligent  data  fusion  [1],  In  this 
program,  three  airborne  techniques:  magnetometry,  LiDAR,  and  orthophotography  were 
used  with  auxiliary  site  data  to  identify  areas  that  are  likely  to  be  contaminated  with 
UXO. 

For  the  RTVS  program,  data  were  collected  from  various  sensor  systems  during  test 
chemical  releases  at  Dugway  Proving  Ground  in  Utah  [2,  3].  The  chemical  clouds 
released  varied  in  direction,  depending  on  the  wind,  and  in  size,  depending  on  the  amount 
of  chemical  released.  Standoff  and  point  detection  systems  were  positioned  to  monitor  a 
test  grid.  The  complete  detection  system  was  comprised  of  five  standoff  IR  imaging 
spectrometers  (AIRIS)  and  twenty-five  point  sensors  (LCD)  collocated  with  twenty-five 
weather  stations  (Airmar).  The  results  are  compared  to  the  legacy  systems  used  at 
Dugway,  which  were  comprised  of  standoff  IR  cameras  (ORION),  standoff  FTIR  sensors 
(MESH),  point  sensors  based  on  photoionization  (ppbRAE),  and  weather  stations 
(PWIDS). 
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As  part  of  this  effort,  NRL  has  used  the  RTVS  test  data  to  develop  data  analysis  and  data 
fusion  methods  for  the  test  grid.  Relevant  components  of  the  data  fusion  framework 
developed  for  wide-area  assessment  of  UXO  have  been  adapted  for  application  to  RTVS 
sensing  systems  and  the  utility  of  several  feature  extraction  techniques  have  been 
evaluated  with  sensor  data. 

The  report  is  structured  as  follows:  Following  the  Introduction,  the  standoff  and  point 
sensors  developed  for  the  RTVS  program  are  briefly  described  in  the  Background 
section,  as  well  as  the  Dugway  sensing  systems.  Next,  strategies  for  implementing  data 
fusion  with  multisensor  systems  are  discussed  in  the  Method  section.  Those  strategies 
with  which  NRL  has  had  the  most  success  are  emphasized.  The  RTVS  testing  and 
evaluation  performed  in  2007  and  2008  is  then  described  in  the  Experimental  section. 
The  description  is  limited  to  that  necessary  for  understanding  the  three-dimensional 
nature  of  the  test  site  and  testing  conditions,  as  more  infonnation  is  available  in  the  test 
summary  reports.  Next,  the  Results  and  Discussion  section  comprises  NRL"s  data  fusion 
analysis  for  the  RTVS  program.  Finally,  Conclusions  are  presented.  A  summary  of  the 
conclusions  is  also  provided  in  the  Executive  Summary. 


4 


2.  BACKGROUND 


2.1.  Sensor  Systems 

Two  sensing  components  were  developed  for  use  in  the  RTVS  test  grid.  One  of  these 
was  a  ground-based  array  of  25  chemical  point  detectors  and  weather  stations  that  were 
dispersed  at  regular  intervals  across  the  grid.  The  ground-based  array  was  developed  by 
ECBC  and  referred  to  as  “SPIDAR,”  short  for  Spectrometric  Point  Ionizing  Detector 
ARray. 

The  chemical  point  detector  selected  for  use  in  SPIDAR  was  the  lightweight  chemical 
detector  (LCD-3™)  available  from  Smiths  Detection,  shown  in  Figure  1.  The  LCD 
detector  is  a  pulse  inlet  ion  mobility  spectrometer  (IMS)  that  samples  the  air  for  chemical 
agents,  toxic  industrial  chemicals,  and  simulants.  Ion  mobility  spectra  in  positive  and 
negative  modes  are  provided  every  five  seconds.  An  intensive  analysis  of  individual 
LCD  responses  to  simulants,  independently  confirmed  through  simultaneous 
quantification  with  gas  chromatography,  led  to  the  development  of  compound-specific 
regression  equations  for  use  in  an  effective  calibration  scheme.  In  addition,  the 
calibration  scheme  employs  reduced  mobility  to  correct  IMS  drift  time  responses  for 
environmental  factors  such  as  temperature  and  pressure. 
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The  weather  station  selected  for  use  in  SPIDAR  was  the  Airmar  Weather  Station  LB  100  / 
PB100  (Airmar),  shown  in  Figure  2.  The  weather  station  provided  meteorological  data 
and  an  integrated  global  positioning  system  (GPS)  receiver.  Ethernet  connectivity  for 
SPIDAR  components  was  provided  by  the  Digi  Port  Server  TS-2  H  MEI  serial  port 
server  and  connected  the  sensors  to  the  SPIDAR  Base  Station  for  data  retrieval  and 
command  and  control.  In  the  first  tests  series  each  of  the  25  stations  communicated 
through  the  center  station  back  to  the  SPIDAR  Base  Station,  whereas  in  the  second  test 
series  each  station  communicated  directly  to  the  SPIDAR  Base  Station. 


Figure  2.  The  Airmar  Weather  Station  PB100. 

The  other  sensing  component  developed  for  the  RTVS  test  grid  was  an  array  of 
irregularly  spaced,  standoff  IR  sensors  developed  by  ECBC  and  Physical  Sciences,  Inc. 
Looking  in  towards  the  test  grid,  the  IR  sensors  were  able  to  observe  the  space  above  the 
grid  to  a  height  over  100  meters.  The  5-member  IR  sensor  grid  consisted  of  a  passive 
multispectral  imaging  system  referred  to  as  “AIRIS,”  for  Adaptive  Infrared  Imaging 
Spectroradiometer.  An  AIRIS  sensor  is  shown  in  Figure  3. 


Figure  3.  Two  views  of  an  AIRIS  sensor. 

An  AIRIS  sensor  is  based  on  a  long-wavelength  IR  (LWIR)  focal  plane  array-based 
camera  that  enables  radiance  measurements  of  sufficient  accuracy  for  species-specific 
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column  densities  of  chemical  clouds  to  be  determined.  A  low-order,  tunable  Fabry-Perot 
etalon  provides  the  spectral  resolution  necessary  to  resolve  structured  absorption  and 
emission  from  molecular  vapors.  Optically,  the  AIRIS  sensors  generated  images  of  256 
x  256  pixels  with  a  field-of-view  of  30  degrees  by  30  degrees.  The  spectral  coverage 
available  was  from  7.9  pm  to  1 1.2  pm  at  roughly  0.1  pm  resolution.  The  outputted  data 
consisted  of  20  wavelengths  acquired  and  processed  every  2  seconds  (0.5  Hz). 

In  addition  to  the  RTVS  sensor  systems,  several  sensor  systems  from  Dugway"s  referee 
system  were  active  during  testing  [2,  3],  Three  infrared  Orion  890VL  Radiometer- 
Tracker  Interface  systems  (i.e.,  infrared  cameras)  were  placed  around  the  grid. 
Radiometer  data  were  recorded  and  processed  to  generate  three-dimensional  positional 
data  and  contour  models  for  detected  simulant  clouds.  MESH,  Inc.  provided  six  scanning 
Fourier-transform  infrared  (FTIR)  chemical  cloud  tracking  systems  (CCTS)  placed 
around  the  grid  similar  to  the  AIRIS  systems.  After  processing,  the  MESH  FTIR  system 
provided  two-dimensional  composite  views  of  simulant  clouds  scaled  to  concentration 
intensity.  The  grid  size  employed  by  the  FTIR  system  was  significantly  larger  and  of  a 
coarser  resolution  than  that  of  AIRIS. 

The  referee  point  chemical  detection  system  consisted  of  multiple  ppbRAE  detectors 
from  RAE  Systems  collocated  with  the  LCD  point  sensors.  In  contrast  to  the  IMS 
technology  used  in  the  LCD  detectors,  the  ppbRAE  systems  were  based  on 
photoionization  sensing  technology  and  were  designed  for  the  detection  of  volatile 
organic  compounds. 

Dugway  also  supplied  meteorological  systems  for  the  test  grid.  These  included  a  32  m 
tower  instrumented  with  temperature  sensors,  a  Sonic  Detection  and  Ranging  (SODAR) 
system,  and  several  Portable  Weather  Information  Display  Systems  (PWIDS)  dispersed 
across  the  test  grid  that  supplied  general  meteorological  information. 
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3.  METHOD 


3.1.  Data  Fusion 

Intelligent  data  fusion  offers  the  possibility  of  viable  multisensory  solutions  built  with 
commercial-off-the-shelf  or  custom  sensing  components  for  both  real-time  and 
retrospective  applications.  Successful  data  fusion  depends  on  more  than  just  increasing 
the  number  of  sensors,  which  often  increases  false  positives  in  lockstep  with  detections 
[4].  Rather,  the  design  of  an  effective  multisensor  detection  system  hinges  on  careful 
choice  of  sensing  hardware,  its  respective  sensor  response  characteristics,  as  well  as  the 
relationship  between  the  sensors  and  the  events  to  be  detected.  Additional  key  challenges 
include  establishing  an  appropriate  frame  of  discernment  for  the  sensing  problem, 
leveraging  decision-level  data  from  available  sensor  algorithms,  and  properly  addressing 
data  disparity  in  fused  systems. 

First,  for  the  purposes  of  this  discussion,  a  multisensor  detection  system  is  one  that  is 
comprised  of  several  component  sensing  elements,  the  outputs  of  which  are  combined 
together  with  signal  processing,  feature  extraction,  and  data  fusion  algorithms  to  produce 
a  quantitative  (or  qualitative)  assessment  of  the  current  sensing  task.  Potential 
applications  of  multisensor  systems  include  chemical  sensing,  explosives  detection,  fire 
detection,  situational  awareness,  and  many  other  sensing  tasks  [5],  There  are  several 
potential  benefits  to  a  multisensor  approach:  by  combining  data  from  multiple, 
orthogonal  sources  of  information,  multisensor  systems  can  demonstrate  improved 
selectivity,  accuracy,  and  reliability  when  compared  to  single  sensor  systems  in  terms  of 
enhanced  detection  capabilities,  reduced  false  positives,  and  decreased  missed  detections. 
The  utilization  of  multiple  sensing  elements  can  also  lead  to  enhanced  system  robustness 
and  fault  tolerance. 

Multisensor  systems  come  with  their  own  set  of  unique  challenges,  however.  The  first  of 
these  is  addressing  data  disparity  in  a  multimodal  sensing  system.  In  a  typical 
multisensor  system,  such  disparity  may  be  as  simple  as  varied  spatial  positioning  among 
sensor  elements,  or  as  complex  as  gross  differences  in  both  the  format  of  the  sensor 
output  and  in  the  fundamental  type  of  information  the  sensor  supplies.  Selecting  optimal 
types,  locations,  or  numbers  of  sensors  is  highly  dependent  on  the  sensing  task  and  may 
be  mitigated  by  external  factors  such  as  deployment  restrictions,  computational 
complexity,  or  available  resources.  A  further  challenge  is  that  merely  adding  sensors  will 
not  necessarily  result  in  improved  system  perfonnance.  For  example,  a  simulation  of  ten 
identical  binary  sensors,  shown  in  Figure  4,  demonstrates  that  a  simple  Boolean  “AND” 
combination  of  the  sensor  outputs  resulted  in  lower  detection  rates  than  those  obtained 
using  a  single  sensor,  while  a  Boolean  “OR”  resulted  in  higher  false  positive  rates.  Thus, 
an  increase  in  the  detection  rate  will  likely  be  accompanied  by  an  unacceptable  increase 
in  false  positives  unless  sensor  data  are  combined  using  intelligent  data  fusion  algorithms 
that  optimize  sensor  perfonnance  in  a  fused  system.  Effective,  reliable  sensors  are 
commercially  available  for  a  multitude  of  sensing  tasks,  but  these  sensors  are  often 
designed  for  single  sensor  environments  and  binary  “threat”  or  “no-threat”  output. 
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Therefore  a  further  challenge  unique  to  multisensor  systems  is  leveraging  decision-level 
data  from  available  sensor  algorithms. 
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Figure  4.  ROC  curve  for  a  single  binary  sensor  overlaid  with  response  characteristics  for  a  simulated 
network  of  ten  identical  sensors  combined  with  Boolean  logic. 

The  development  of  a  multisensor  system  is  an  iterative  process  and  includes  a  number  of 
key  ingredients,  each  of  which  are  partially  dependent  on  the  others.  The  first  of  these  is 
defining  a  frame  of  discernment,  which  makes  explicit  the  sensing  tasks  to  be 
accomplished.  Next  is  the  selection  of  sensors  suitable  for  the  chosen  frame  of 
discernment.  Different  sensing  modalities  should  be  included  to  provide  complementary 
information  that  addresses  challenges  posed  by  the  frame  of  discernment  or  limitations 
posed  by  sensors  of  a  single  modality.  Issues  with  available  sensors  may  force  a 
refinement  in  the  frame  of  discernment.  Next  is  the  development  of  relevant  feature  sets 
from  sensor  data.  Commercial  sensors  with  smart  algorithms  should  be  properly 
leveraged  for  optimal  benefit.  Finally,  features  and  data  should  be  integrated  with  data 
fusion  algorithms.  NRL  has  found  that  a  modular  design  to  system  architecture,  together 
with  a  tiered  approach  to  algorithmic  analysis,  to  be  most  useful  for  developing  effective 
multisensor  systems. 

In  a  multisensory  system,  data  disparity  can  manifest  in  several  ways.  Individual  sensing 
units  may  be  spatially  separated  from  one  another  within  a  monitored  space,  or  may 
display  temporal  disparity,  acquiring  readings  at  different  times  or  with  different 
frequencies.  In  these  cases,  data  registration  techniques  must  be  employed.  More  severe 
are  situations  involving  different  sensing  modalities,  in  which  there  is  disparity  between 
the  fonn  of  the  sensor  output  (for  example,  the  combination  of  binary  and  continuous 
output  sensors),  or  in  the  type  of  information  that  is  being  presented  (for  example,  the 
combination  of  light  sensors  and  microphone  outputs).  In  these  cases,  data  processing 
and  feature  extraction  techniques  must  be  used  to  extract  relevant  information  from  the 
sensor  outputs  to  which  the  data  fusion  algorithms  can  be  tailored. 

Approaches  to  data  fusion  generally  fall  into  one  of  the  following  categories:  1)  heuristic/ 
rule  based  algorithms  that  involve  construction  of  a  logical  decision  tree  that  encodes 
known  relationships  between  sensors  and  detectable  events,  2)  pattern  recognition 
algorithms  that  operate  on  data  vectors  consisting  of  aggregate  sensor  and  algorithm 
output,  3)  Bayesian  theoretic  algorithms  that  take  advantage  of  knowledge  regarding  the 
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statistical  relationship  between  collected  sensor  outputs  and  detectable  events,  4) 
Dempster-Shafer  theoretic  algorithms  that  are  based  on  subjective  assessments  of  belief 
that  specific  sensor  outputs  convey,  and  5)  hybrid  algorithms  consisting  of  some 
combination  of  these  approaches. 

For  instance,  specific  heuristic  rules  can  come  from  theoretical  or  empirically  derived 
expert  knowledge  regarding  sensor  response  characteristics  and  sensor-to-sensor 
relationships.  While  a  heuristic  approach  is  simple  computationally  and  allows  for 
detailed  input  of  known  relationships  between  sensor  responses  and  detectable  events,  it 
also  leads  to  complexity  in  the  design  process,  inability  to  capture  hidden  relationships, 
and  to  a  lack  of  flexibility  that  makes  heuristic  rules  difficult  to  implement  with  data  sets 
that  vary  in  format  and  composition  over  time. 

Pattern  recognition  techniques  include  classification  algorithms  such  as  linear 
discriminant  analysis  (LDA)  and  probabilistic  neural  networks  (PNN),  as  well  as 
numerous  others  [6].  These  techniques  can  act  as  data  fusion  algorithms  when  they  are 
applied  to  a  vector  of  aggregated  sensor  responses.  In  and  of  themselves,  they  have 
limited  capabilities  to  address  data  disparity,  but  can  represent  a  potent  means  of 
discovering  hidden  relationships  between  sensor  data  and  detectable  events  given 
appropriate  data  preprocessing  and  proper  training  with  experimental  data. 

Bayesian  reasoning  and  Bayesian  belief  network  architectures  are  powerful  tools  for 
inductive  reasoning  within  a  probabilistic  framework  [7,  8].  The  core  of  the  Bayesian 
approach  was  fonnulated  from  Bayes"  rule,  shown  in  eq.  (1),  which  states  that  for  a 
hypothesis,  H,  and  evidence,  E,  the  probability  of  the  H,  conditioned  on  E  is  given  by 

I  P(e\h) 

P(h\e)=p(H)x -  (1) 

P(E) 

The  first  term,  p(H),  is  the  prior  probability  of  H  and  represents  knowledge  of  H  before 
observing  evidence  E.  The  second  term  is  the  normalized  likelihood  and  represents 
knowledge  of  how  likely  it  is  to  observe  E,  given  H.  Multiplication  of  these  two  terms 
provides  the  posterior  probability  of  H,  given  observed  evidence  E.  If  multiple  lines  of 
evidence,  E  =  {ei,  e2,  e3,...  en},  are  available,  the  joint  distributions  must  be  considered, 
p({ei,  e2,  e3,...  en}|H)  and  p({ei,  e2,  e3,...  en}). 

Fortunately,  in  many  cases  calculating  the  joint  distribution  of  several  variables  is  made 
simpler  by  examining  the  causal  relationships  among  the  variables  and  identifying  those 
that  are  independent.  A  specification  of  these  relationships  and  the  conditional 
probability  values  associated  with  them  forms  what  is  known  as  a  Bayesian  belief 
network.  A  further  simplified  “naive”  Bayesian  approach  is  to  assume  independence 
amongst  each  line  of  evidence,  as  in  eq.  (2). 
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P(  ^,e2,e3,  .  ef)  \H)=y\p(e\H)  (2) 

1=1 

The  initial  prior  probability  for  H  can  be  updated  to  a  posterior  probability  of  H  by 
consecutively  multiplying  it  by  the  normalized  likelihood  of  each  available  line  of 
evidence,  p(ei|H)  / p(e;),  for  i  e  {1  ...  n). 

Briefly,  Dempster-Shafer  theory  can  be  described  as  a  generalization  of  Bayes  theory  in 
which  observed  evidence  can  support  not  only  specific  hypotheses,  but  also  sets  of 
hypotheses  [5,  9].  Dempster-Shafer  data  fusion  has  been  utilized  in  applications  such  as 
land  cover  classification,  machine  vision,  and  medical  diagnoses  [10,  11,  12].  The 
approach  is  attractive  for  problems  involving  evidence  pooling  from  multiple  sources 
because  it  does  not  require  complete  specification  of  the  underlying  conditional 
probabilities  and  it  allows  for  assignment  of  a  degree  of  belief  to  a  specific  hypothesis 
without  necessarily  assigning  any  belief  to  the  negation  of  that  hypothesis.  These 
properties  enable  Dempster-Shafer  frameworks  to  address  evidence  associated  with 
subjective  uncertainty  in  a  more  satisfying  manner  than  a  Bayesian  approach  while 
simultaneously  retaining  the  ability  to  incorporate  evidence  associated  with  objective 
uncertainty  [13]. 

In  Dempster-Shafer  theory,  the  evidence  observed  from  a  given  sensor  response  provides 
support  for  one  or  more  of  the  elements  of  the  frame  of  discernment  including, 
potentially,  supersets  combining  two  or  more  elements.  A  unit  probability  mass  is 
apportioned  across  the  elements  (and  sets  of  elements)  for  which  the  sensor"s  response 
provides  support.  The  amount  of  mass  assigned  to  any  given  focal  element  is 
representative  of  the  relative  amount  of  certainty  with  which  the  sensor  response  can 
make  that  declaration.  Mass  assigned  to  a  superset  of  all  unique  elements  in  the  frame  of 
discernment  represents  the  degree  to  which  the  data  are  uncertain  in  providing  support  for 
any  particular  element. 

Probability  mass  assignments  from  multiple  information  sources  are  pooled  according  to 
Dempster"s  rule  of  combination,  which  amounts  to  a  calculation  of  the  cross  products  of 
the  probability  mass  assignments  for  each  data  source.  Each  cross  product  is  assigned  to 
a  hypothesis  that  represents  the  intersection  of  the  two  component  probability  masses 
used  to  generate  it.  Cross  products  assigned  to  the  same  hypothesis  are  summed, 
resulting  in  a  series  of  unique  output  hypotheses  with  corresponding  probability  mass 
assignments.  If  any  cross  products  result  from  conflicting  hypotheses,  they  are  removed 
from  the  set  and  the  remaining  masses  are  normalized  to  sum  to  unit  mass.  Such 
renormalization  has  the  effect  of  redistributing  the  mass  assigned  to  conflict 
proportionally  across  the  remaining  hypotheses. 

In  the  example  shown  in  Figure  5,  evidence  from  two  sensors  providing  support  for 
varying  elements  in  the  frame  of  discernment  (labeled  A,  B,  C,  and  D)  is  pooled 
according  to  Dempster"s  rule.  The  net  effect  is  the  assignment  of  most  of  the  probability 
mass  to  element  A  (=0.56),  a  reduction  of  those  assigned  to  the  set  of  elements  A  and  B 
(=0.24)  and  A  and  C  (=0.14),  as  well  as  a  reduction  of  the  mass  assigned  to  uncertainty 
(=0.06). 
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Figure  5.  Hypothetical  example  of  Dempster-Shafer  fusion  of  evidence  provided  by  two  disparate  sensors 


4.  EXPERIMENTAL 


As  part  of  the  program,  two  field  tests  were  conducted  to  evaluate  the  RTVS  detection 
systems  (LCD  and  AIRIS)  and  contrast  their  performance  against  the  legacy  Dugway 
systems  (ppbRAE  and  MESH).  Both  test  series  were  conducted  at  the  US  Army"s 
Dugway  Proving  Ground  in  Utah  and  consisted  of  successive  releases  of  various 
chemical  agent  simulants  under  several  dissemination  scenarios.  Two  different  test  grids 
were  used  for  the  field  tests.  The  first  test  series  (RTVS  "07)  was  performed  at  Dugway"s 
V-Grid,  during  the  nights  of  17  -  21  September  2007  [2],  The  second  test  series  (RTVS 
"08)  was  at  Dugway"s  S-Grid,  during  the  nights  of  09  -  19  June  2008  [3].  The  layout  for 
the  detection  systems  was  similar  for  both  test  series.  Point  sensors  with  collocated 
weather  stations  were  distributed  near  ground  level  at  locations  across  a  square  grid.  The 
standoff  sensors  were  distributed  roughly  symmetrically  around  the  square  grid  of  point 
sensors,  but  at  distance  of  1  -  3  km  from  grid  center.  Data  acquired  by  the  sensors  were 
relayed  wirelessly  to  an  on-site  command  post  for  processing  and  logging.  Dissemination 
locations  for  the  simulants  were  also  located  more  than  1  km  distant  from  grid  center,  but 
were  clustered  near  the  central  axis  of  the  test  grid.  Test  scenarios  were  conducted  by 
Dugway  personnel.  A  typical  testing  procedure  consisted  of  10  -  20  minutes  of 
background  data  collection  followed  by  the  release  of  one  (or  more)  simulants  and 
continued  data  acquisition.  The  cessation  of  data  acquisition,  after  anywhere  from  15  - 
60  minutes,  constituted  the  end  of  an  individual  test  scenario.  Set  up  for  the  next  test 
scenario  was  conducted  after  detennination  of  a  safe  all-clear  environment  by  Dugway 
testing  personnel. 

4,1,  RTVS ’07  Test  Series 

The  first  RTVS  test  series  was  conducted  17-21  September  2007  on  Dugway"s  V-Grid. 
The  layout  of  the  grid,  RTVS  and  Dugway  detection  systems,  weather  stations,  and 
simulant  dissemination  locations  are  shown  in  Figure  6.  The  core  of  V-Grid  was  a  square 
of  1  km  per  side  rotated  approximately  15  degrees  clockwise  from  true  north.  The  25 
RTVS  point  sensors  (SPIDAR  LCD"s)  were  distributed  at  a  height  of  1  meter  equitably  in 
nine  rows  across  this  core  grid  (see  Figure  7  for  positions  and  labels  of  the  LCD  sensors). 
SPIDAR  Ainnar  weather  stations  were  also  located  coincident  with  all  LCD  detectors  at 
a  height  of  about  1.5  meters.  The  five  RTVS  standoff  sensors  (AIRIS)  were  positioned 
2.7  km  from  grid  center.  Wide  area  AIRIS  detectors  were  placed  (see  Figure  6)  at  the 
Northwest,  East,  and  South  positions  while  tomographic  AIRIS  detectors  were  placed  at 
the  Northeast  and  Southeast  positions. 
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Figure  6.  Target  V-Grid,  Dugway  Proving  Ground,  UT.  AIRIS  -  Adaptive  Infrared  Imaging 
Spectroradiometer  sensors;  IR  -  Orion  infrared  cameras;  Diss.  Pts  -  dissemination  points  for  chemical 
simulants;  FTIR  -  MESH  scanning  Fourier-transform  infrared  (FTIR)  sensors;  LCD  -  SPIDAR 
Lightweight  Chemical  Detectors;  GC  -  grid  center;  Det  Pts  -  detonation  points;  PWID  -  Portable 
Weather  Information  Display  System;  BB  -  reference  blaekbody  reflector  for  AIRIS;  T  -  tomographic 
AIRIS;  WAD  -  wide  area  detector  AIRIS;  (not  shown)  -  ppbRAE  point  sensors  at  various  LCD 
locations 


RIS 


BB-N 


orthwest  (2  -  W) 


FTIR-5 


260° 


G  R 


309OOOm.E 


•54 


# 

Alternate  Det  Pts 

• 

Diss  Pts 

0 

PWID 

G 

LCD 

• 

AIRIS 

« 

Scanning  FTIR 

• 

GC 

A 

V-Bunker 

Kl 

IR  Camera 

14 


Q  LCD  location  with  generator  A 

212.1  m=  695  7  ft  S 

269.3  m=  883.3  ft  NORTH 

I 


From  North 

Figure  7.  Positions  and  labels  of  LCD  detectors  and  Airmar  weather  stations 
on  V-Grid  for  the  RTVS  ’07  test  series. 


For  the  Dugway  standoff  detection  systems,  five  scanning  FTIR  systems  (MESH)  were 
placed  (see  Figure  6)  roughly  coincident  with  the  AIRIS  systems,  but  located  generally 
nearer  to  grid  center.  A  total  of  three  Orion  IR  cameras,  at  six  locations,  were  used.  Five 
of  the  locations  were  with  the  scanning  FTIR  systems,  and  the  sixth  was  located 
northwest  of  the  core  grid  roughly  halfway  between  IR-1  and  IR-5.  Thirteen  Dugway 
point  sensing  systems  (ppbRAE)  were  placed  coincident  with  the  RTVS  point  sensors,  of 
these  only  seven  were  calibrated  and  used  during  any  given  test.  The  selection  of  RTVS 
point  sensor  locations  at  which  to  place  coincident  ppbRAE  sensors  was  varied  for  each 
testing  day  (see  [2]).  Five  PWIDS  weather  stations  were  placed  in  the  core  grid.  One 
PWIDS  was  located  at  grid  center  and  the  other  four  were  located  at  the  midpoints  of  the 
1  km  sides  of  the  core  grid  square. 

Only  one  chemical  simulant,  triethyl  phosphate  or  TEP,  was  released  during  the  RTVS 
"07  test  series.  A  total  of  twelve  release  scenarios  were  conducted  with  the  amount  of 
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TEP  released  varying  from  10  to  160  kg.  Dissemination  was  performed  using  explosive 
detonation  at  three  locations:  northwest  of  the  test  grid  2-2.5  km  from  grid  center  (two 
tests),  southeast  of  the  test  grid  2  -  2.5  km  from  grid  center  (seven  tests),  and  at  a  third 
location  southwest  of  the  grid  (five  tests).  Liquid  TEP  in  plastic  jugs  was  placed  on  C4 
explosive  and  a  blasting  cap  was  used  to  achieve  detonation,  which  both  vaporized  the 
simulant  and  pushed  it  high  into  the  air.  Table  A1  in  the  Appendix  gives  a  summary  of 
the  tests  run  during  RTVS07. 

Data  acquired  by  the  sensing  systems  was  transmitted  wirelessly  from  the  location  of 
acquisition  to  a  command  post  located  about  1  km  south  of  the  core  grid.  The  hardware 
was  specified  by  Dugway  for  secure  wireless  transmissions.  For  the  LCD  point  sensors, 
acquired  data  were  minimally  processed  by  the  sensors  themselves  before  secure 
transmission  in  real-time  to  the  command  post.  There,  software  developed  by  the 
SPIDAR  team  running  on  a  PC-grade  workstation  processed  the  data  for  heads-up 
display  and  logged  it  for  later  analysis.  For  the  AIRIS  sensors,  the  volume  of  acquired 
data  was  far  in  excess  of  the  capacity  of  the  wireless  system.  Thus,  AIRIS  data  were 
processed  and  stored  by  a  PC-grade  notebook  located  at  each  acquisition  site.  An 
abbreviated  version  of  the  data  was  transmitted  in  real-time  to  the  command  post  where 
software  developed  for  AIRIS  running  on  PC-grade  workstations  processed  the  data  for 
heads-up  display.  The  heads-up  display  provided  2-D  horizon  views  from  each  reporting 
AIRIS  sensor  and  also  estimates  of  the  three  dimensional  spatial  extent  of  the  simulant 
cloud  and  its  concentration  profile. 

Data  from  the  LCD  sensors  and  Airmar  weather  stations  were  made  available  in  one  large 
ASCII  (text)  file.  Individual  sensors  were  identified  by  location  (northing,  easting)  and 
time  stamp.  Post-processing  was  employed  with  the  AIRIS  data  to  perfonn  three- 
dimensional  tomographic  reconstruction  of  the  simulant  clouds.  Data  were  reconstructed 
into  a  cube  comprised  of  256  x  256  x  256  voxels  (i.e.,  a  3-D  volume  element  analogous 
to  2-D  pixels.)  Ground  level  in  the  reconstructed  data  cube  corresponded  to  all  voxels  in 
z-bin  128  (i.e.,  i^yx  128  for  (x,  y)  e  1:256).  Post-processed  data  were  made  available 
as:  (1)  JPEG  images  of  the  2-D  views  from  each  of  the  five  AIRIS  sensors,  (2)  a  series  of 
JPEG  images  showing  the  3-D  simulant  cloud  against  easting,  northing,  and  vertical  z 
coordinate  axes  at  various  time  stamps,  and  (3)  as  a  binary  file  containing  the  detected 
concentrations  in  each  voxel  of  the  data  cube  at  all  reconstructed  times. 

4.2.  RTVS ’08  Test  Series 

The  second  RTVS  test  series  was  conducted  09-19  June  2008  on  Dugway"s  S-Grid. 
The  layout  of  the  grid,  RTVS  and  Dugway  detection  systems,  weather  stations,  and 
simulant  dissemination  locations  are  shown  in  Figure  8.  The  core  of  S-Grid  was  a  square 
of  1  km  per  side  rotated  approximately  5  degrees  clockwise  from  true  north.  The  25 
RTVS  point  sensors  (SPIDAR  LCD"s)  were  distributed  at  a  height  of  1  meter  across  this 
core  grid  (see  Figure  8)  as  in  the  earlier  test  series  (see  Figure  7).  SPIDAR  Airmar 
weather  stations  were  also  located  coincident  with  all  LCD  detectors  at  a  height  of 
approximately  1.5  meters.  The  five  RTVS  standoff  sensors  (AIRIS)  were  positioned  2.7 


16 


km  from  grid  center.  The  distribution  of  the  AIRIS  systems  around  grid  center  was  not 
entirely  symmetric.  A  gap  in  coverage  occurred  northeast  of  the  grid,  opposite  AIRIS-3. 

For  the  Dugway  standoff  detection  systems,  five  scanning  FTIR  systems  were  placed  (see 
Figure  8)  around  the  core  grid,  however  only  one  (FTIR-2)  was  roughly  coincident  in 
field  of  view  with  an  AIRIS  system  (AIRIS-3).  A  total  of  six  Orion  IR  cameras  were 
used,  five  of  which  were  located  coincident  with  the  scanning  FTIR  systems,  and  the 
sixth  was  located  northwest  of  the  core  grid  on  the  grid  diagonal.  Fifty  Dugway  point 
sensing  systems  (ppbRAE)  were  placed  coincident  with  the  RTVS  point  sensors,  two  per 
LCD  location.  The  paired  ppbRAE  systems  were  placed  on  a  single  mast  at  heights  of  1 
and  6  meters.  Five  PWIDS  weather  stations  were  placed  in  the  core  grid.  One  PWIDS 
was  located  at  grid  center  and  the  other  four  were  located  at  the  midpoints  of  the  1  km 
sides  of  the  core  grid  square. 

Three  chemical  simulants  were  released  during  the  RTVS  ”08  test  series:  triethyl 
phosphate  (TEP),  methyl  salicylate  (MeS),  and  acetic  acid  (AA).  A  total  of  29  release 
scenarios  were  conducted:  fifteen  TEP  releases  of  30  -  150  kg,  seven  MeS  releases  of  30 
-  150  kg,  two  MeS/TEP  joint  releases  of  30  and  75  kg  each,  four  AA  releases  of  30  -  150 
kg,  and  one  “blank”  scenario  consisting  of  a  dust  cloud  without  any  simulant. 
Dissemination  was  performed  using  explosive  detonation  (as  in  RTVS  ”07)  at  two  of 
three  northern  dissemination  sites  (eight  tests  total  at  N2  and  N3),  all  of  the  southern 
grids  (17  tests  total  at  SI,  S2,  and  S3),  and  at  an  additional  west  center  location  used  for 
stack  releases  (four  tests).  During  a  stack  release,  liquid  simulant  was  continuously 
pumped  through  a  shaft  attached  to  a  crane  and  then  vaporized  about  20  meters  above 
ground.  A  stack  release  typically  lasted  fifteen  minutes.  Table  A2  in  the  Appendix  gives 
a  summary  of  the  tests  run  during  RTVS08. 

Data  acquired  by  the  sensing  systems  was  transmitted  wirelessly  from  the  location  of 
acquisition  to  a  command  post  located  about  1  km  south  of  the  core  grid.  For  RTVS  ”08 
however,  Dugway  provided  a  different  hardware  system,  Cisco  1300  series  radios,  for 
secure  wireless  transmissions.  The  data  assimilation,  display,  and  logging  systems 
employed  for  the  SPIDAR  LCD  point  sensors,  Ainnar  weather  stations,  and  AIRIS 
standoff  sensors  were  essentially  identical  to  RTVS  "07.  For  RTVS  "08,  the  LCD  data 
were  post -processed  to  remove  transient  duplicate  points.  Data  were  made  available  in 
the  same  text-based  format.  AIRIS  data  were  post-processed  as  for  RTVS  ”07,  but  the 
reconstructed  3-D  data  cubes  was  made  available  in  a  sparse  matrix  fonnat,  which  greatly 
reduced  the  file  sizes. 
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Figure  8.  Target  S,  Dugway  Proving  Ground,  UT.  AIRIS  -  Adaptive  Infrared  Imaging  Spectroradiometer 
sensors;  IR  -  Orion  infrared  cameras;  Diss.  Pts  -  dissemination  points  for  chemical  simulants; 
NNWDA  -  north  northwest  dissemination  area;  SSEDA  -  south  southeast  dissemination  area;  FTIR  - 
MESH  scanning  Fourier-transform  infrared  (FTIR)  sensors;  LCD  -  SPIDAR  Lightweight  Chemical 
Detectors;  PWID  -  Portable  Weather  Information  Display  System;  BBR  -  reference  blackbody 
reflector  for  AIRIS;  (not  shown)  -  two  ppbRAE  point  sensors  at  each  LCD  locations;  numbers  126 
through  702  are  road  numbers. 
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5.  RESULTS  AND  DISCUSSION 


5.1.  RTVS ’07  Test  Series 

The  September  2007  RTVS  test  series  was  the  first  time  the  RTVS  sensor  systems  had 
been  assimilated  and  field  tested  as  a  group.  During  testing  at  Dugway,  winds  were  light 
and  variable,  which  made  simulant  releases  difficult.  Acquired  data  were  also  limited 
due  to  issues  with  the  communications  network.  As  a  whole,  the  test  series  served 
primarily  as  a  necessary  shakeout  for  the  newly  developed  data  acquisition, 
communications  systems,  and  testing  methods.  Areas  for  improvement  were  identified 
and  test  data  were  made  available  for  preliminary  data  fusion  studies  exploring  data 
registration  and  feature  selection  strategies  for  the  RTVS  data. 

During  testing  the  SPIDAR  LCD  detectors  had  difficulties  communicating  in  real-time 
with  the  command  post  over  the  wireless  network.  The  source  of  the  problem  was 
suspected  to  be  a  strong  radar  source  that  knocked  out  the  system  during  setup  and 
caused  it  to  run  at  a  reduced  capacity  for  the  remainder  of  the  test  series.  Table  1 
indicates  from  which  LCD  detectors  data  were  received  for  each  of  the  twelve  tests.  A 
total  of  six  to  seven  LCD  detectors  reported  data  consistently  for  every  test,  while  one  to 
two  additional  detectors  reported  partial  data  for  some  tests.  Therefore,  data  were 
obtained  from  only  about  a  third  of  the  LCD  detectors.  LCD  data  were  supplied  in  text 
format  as  chemical  type  and  detected  concentration  as  a  function  of  time.  Figure  9  shows 
the  detected  concentrations  as  a  function  of  time  of  four  LCD  detectors,  those  at  SW 
Corner,  Center,  NE  1,  and  NE  3,  to  two  similar  30  kg  TEP  simulant  releases  from  the 
southwest.  These  were  the  only  LCD  detectors  to  respond  during  these  tests  and  transmit 
data.  The  initial  strong  and  persistent  simulant  signals  observed  at  the  SW  Corner  closest 
to  the  release  point  have  been  dispersed  or  have  not  reached  far  across  the  grid,  most 
likely  due  to  weak  winds.  The  final  test  of  the  series,  test  #13T160,  a  release  of  160  kg 
TEP  from  the  southeast,  better  demonstrates  the  potential  for  a  grid  of  point  sensors  such 
as  the  LCD  detectors.  The  responses,  shown  in  Figure  10,  follow  the  simulant  cloud 
from  the  southeast  as  it  moves  across  the  grid  and  dissipates. 


Figure  9.  Examples  of  LCD  readings  versus  time  for  two  similar  30  kg  TEP  releases  from 
southwest  of  the  grid. 
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Table  1.  Limits  of  data  obtained  for  LCD  detectors  on  V-Grid  for  the  RTVS  ’07  test  series 
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Figure  10.  Example  of  LCD  readings  versus  time  for  a  160  kg  TEP  release  from  southeast 
of  the  grid. 

In  general,  the  AIRIS  standoff  detectors  responded  well  to  the  TEP  releases.  The 
response  of  the  AIRIS  detectors  approximately  10  minutes  after  the  release  of  100  kg  of 
TEP  simulant  is  shown  in  Figure  1 1  and  Figure  12.  AIRIS  data  were  supplied  in  binary 
and  image  formats.  Figure  11  is  an  example  of  image  data  displaying  five  real  time  2D 
views  from  the  AIRIS  detectors.  The  2D  view  displays  the  cross  sectional  area 
perpendicular  to  that  AIRIS  instrumenf's  field  of  view  with  the  horizon  line  roughly 
bisecting  the  middle  of  the  image.  In  the  2D  view,  detected  simulants  were  shown  as 
colored  pixels  while  an  estimate  of  the  concentration  detected  was  supplied  in  the  lower 
left  of  the  image.  Figure  12  shows  an  example  of  3D  image  data  constructed 
retrospectively  from  the  individual  2D  views  using  computational  tomography  [14].  The 
figure  displays  the  reconstructed  simulant  cloud  together  with  its  projections  in  the 
northing  (a),  easting  ( v),  and  horizontal  (z)  planes.  These  projections  also  show  the 
concentration  of  the  column  density.  The  entire  reconstructed  data  cube  consisting  of 
256  x  256  x  256  voxels  as  a  function  of  time  was  supplied  in  a  binary  file.  Each  voxel 
contained  the  concentration  for  that  location  in  space  and  step  in  time.  These  data  were 
used  for  data  fusion  development. 


SENSOR  1 

9/19/2007  11:50:11.000 
Mean  CD=  1547.97  ArTemp=  8 
Mass  Estimated=  16.0030 
.SN001  _Cube0709 1 9_0358.acd 


Figure  11.  Two-dimensional  sensor  views  of  the  AIRIS  detectors’  responses  approximately  10  minutes  after  the 
release  of  100  kg  TEP.  The  red  pixels  indicate  the  extent  of  the  detected  simulant  cloud.  Sensors  1,  2, 
and  4  were  the  newer  wide-area  type.  Sensors  3  and  5  were  the  older  tomographic  type. 
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Figure  12.  Tomographically  constructed  three-dimensional  view  of  the  AIRIS  detectors’  responses 
approximately  10  minutes  after  the  release  of  100  kg  TEP.  Ground  level  is  shown  in  blue  shading  with 
projections  in  lavender  shading. 

As  for  weather  data  during  RTVS  "07  testing,  the  communications  issues  meant  that  even 
fewer  SPIDAR  Airmar  weather  stations  than  LCD  detectors  were  able  to  report  to  the 
command  post.  The  Ainnar  weather  stations  were  administered  by  the  SPIDAR  team 
and  transmitted  data  simultaneously  with  the  LCD  systems.  Only  the  weather  station 
located  at  the  Center  location  reported  data  consistently  for  the  test  series.  Partial  data 
was  received  from  a  few  other  Airmar  weather  stations  (one  to  four)  for  portions  of  some 
of  the  tests.  Measured  weather  data  were  temperature,  pressure,  relative  humidity,  wind 
speed,  and  wind  direction,  but  were  only  compiled  during  active  data  acquisition  (i.e., 
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during  testing).  The  recorded  pressures  were  not  calibrated  correctly  for  the  height  above 
sea  level  at  V-Grid  and  required  correction  using  the  formula  given  in  eq.  (3), 


Pat(h)=  P 


,  \  5.2  5  6  1 

2  8  8—  0.00  6  5/z 

V  288  y 


(3) 


Here  h  is  the  height  above  sea-level  (i.e.,  altitude)  in  meters  and  Pstation  is  the  barometric 
pressure  measured  at  the  weather  station.  The  altitude  of  the  center  station  was  1305.97 
meters.  Pait  is  the  resulting  altitude-corrected,  barometric  pressure. 

A  much  better  weather  dataset  was  provided  by  the  five  Dugway  PWIDS  weather 
stations.  Measured  weather  data  were  of  the  same  type:  temperature,  pressure,  relative 
humidity,  wind  speed,  and  wind  direction,  as  the  Airmar  weather  stations,  but  data  were 
supplied  at  intervals  of  10  seconds  for  the  entire  two-week  testing  period.  Figure  13 
displays  the  temperature,  pressure,  and  relative  humidity  data  obtained  from  the  five 
PWIDS  weather  stations  overlaid  together  with  its  maximum  variance  as  a  function  of 
time  for  the  week  of  data  taking,  09/16/2007  -  09/21/2007.  The  data  vary  significantly  as 
a  function  of  time  over  the  day  /  night  cycle  and  given  weather  patterns.  However  for  a 
given  time  selection,  these  data  are  remarkably  consistent  between  PWIDS  stations, 
indicating  that  there  was  little  variation  in  temperature,  pressure,  or  relative  humidity 
with  respect  to  grid  location,  at  least  at  ground  level. 

Because  the  Airmar  stations  were  less  sophisticated  instruments  and  had  not  been  used 
before,  there  was  some  question  as  to  the  efficacy  of  their  data.  To  test  this,  weather  data 
acquired  from  the  Airmar  stations,  where  available,  were  compared  to  those  obtained 
from  PWIDS.  Figure  14  shows  a  comparison  of  the  temperature,  pressure,  and  relative 
humidity  data  from  the  PWIDS  and  Airmar  weather  stations  collocated  at  grid  center 
during  the  week  of  testing.  The  data  are  sufficiently  consistent  with  each  other,  given  the 
difference  in  cost  between  the  two  types  of  weather  stations. 
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Figure  13.  Overlays  of  temperature,  pressure,  and  relative  humidity  weather  data  from  the  five  PWIDS  weather 
stations  for  the  dates  09/16/2007  -  09/21/2007. 
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Date 


Figure  14.  Comparison  of  temperature,  pressure,  and  relative  humidity  weather  data 
from  the  PWIDS  (magenta  line)  and  Airmar  (blue  crosses)  weather  stations 
collocated  at  grid  center.  The  red  circles  mark  the  start  times  for  the  13  tests. 


5.2.  RTVS  ’08  Test  Series 

The  June  2008  RTVS  test  series  was  much  more  successful  than  the  2007  test  series  in 
terms  of  the  quantity  and  quality  of  data  collected  by  the  RTVS  sensor  systems  and  the 
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number  and  variety  of  tests  performed.  Environmental  conditions  on  Target  S  were  more 
stable,  which  made  possible  more  simulant  releases.  Issues  with  a  new  wireless 
communications  network  supplied  by  Dugway  were  quickly  resolved.  A  larger  data  set 
of  higher  quality  was  acquired  and  made  available  for  data  fusion  algorithm 
development. 

Between  the  2007  and  2008  test  series,  the  SPIDAR  team  had  tested  a  new  access  point 
wireless  communications  system  in  the  field  at  Dugway  in  order  to  resolve  the  issues  that 
had  compromised  data  acquisition  for  the  LCD  detectors  and  Airmar  weather  stations. 
However,  the  communications  hardware  selected  and  tested  by  SPIDAR  did  not  meet 
evolving  security  requirements  for  wireless  systems  at  Dugway  in  time  for  the  2008  test 
series.  Therefore,  Dugway  provided  an  alternative,  but  slower,  point  to  multi-point 
wireless  system  for  sensor  communications.  During  test  set  up  and  shakedown,  several 
communications  issues  developed  with  the  SPIDAR  systems  due  to  the  interaction  of 
low-level  driver  software  supplied  by  Smiths  Detection  with  the  less  capable  Dugway 
wireless  system.  The  SPIDAR  resolved  the  issues  by  modifying  their  software  to  more 
closely  supervise  wireless  transmissions,  ensuring  that  all  data  from  all  detectors  was 
transmitted  sequentially  to  the  base  station.  By  the  end  of  the  first  week,  more  than  20 
LCD  detectors  were  reporting  completely,  although  this  was  achieved  by  sacrificing  the 
data  from  the  weather  stations  to  minimize  the  data  traffic  on  the  network.  An  additional 
issue  was  that  the  LCD  data  were  observed  to  have  occasional  dropouts,  that  is,  gaps  in 
the  time  series  data,  which  were  due  to  a  software  problem  that  was  found  and  corrected. 

Once  the  communications  issues  were  resolved,  nearly  all  the  LCD  detectors  were 
reporting  valid  streams  of  data  during  testing.  The  same  single  file,  text-based  data 
format  was  used  for  data  logging,  though  without  the  file  locking  issues  observed  in 
2007.  Ligure  15  shows  the  responses  of  the  LCD  detectors  (blue)  to  an  explosive  release 
of  150  kg  of  TEP.  The  responses  are  shown  as  concentrations  (mg/m3)  versus  time  after 
release  (minutes).  The  placement  of  the  graphs  in  the  figure  corresponds  to  the  locations 
of  the  LCD  detectors  on  the  test  grid.  As  a  group,  the  response  of  the  LCD  detectors  is 
consistent  with  that  of  a  simulant  cloud  drifting  across  the  grid  from  the  northwest  (upper 
left)  to  the  southeast  (lower  right).  In  Ligure  15,  the  numbers  in  the  upper  left  of  the 
graphs  indicate  the  onset  of  detection  in  minutes  for  the  LCD  at  that  location.  Times  for 
the  leftmost  graphs  were  1.2  min,  6.9  min,  and  3.5  min  for  locations  NW-Comer,  NW-1, 
and  NW-3,  respectively.  (See  Ligure  7  for  LCD  location  identifiers.)  Times  for  the 
rightmost  graphs  were  7.0  min  (N-Edge),  2.6  min  (NE-2),  4.1  min  (NE-4),  and  6.7  min 
(E-Edge).  Linally,  the  LCD  at  SE-Corner,  located  furthest  from  the  simulant  source, 
detected  the  release  after  22.8  min.  While  the  general  results  were  consistent  with  a 
simulant  cloud  drifting  across  the  grid  from  the  northwest  to  the  southeast,  the  variations 
in  incipient  LCD  detection  times  and  missing  detections  indicated  that  the  cloud  dispersal 
was  not  linear  across  the  grid  at  ground  level  despite  relatively  uniform  wind  speeds  and 
directions. 
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Figure  15.  Comparison  of  LCD  (blue)  and  AIRIS  (red)  responses  versus  time  after  release  for  a  150  kg  explosive 
release  of  TEP  (test  #05T150).  The  placement  of  the  graphs  corresponds  to  the  LCD  grid  locations. 
The  numbers  indicate  the  start  times  for  TEP  detection  for  LCD  (left)  and  AIRIS  (right). 

The  performance  of  the  AIRIS  systems  during  the  2008  test  series  was  similar  to  that 
of  2007.  However,  during  the  first  week  of  testing,  one  of  the  AIRIS  sensors  failed, 
followed  by  another  unit  during  the  second  week.  The  format  of  the  image  data  was 
identical  to  that  of  2007.  The  binary  data  format  was  improved  by  using  a 
compression  algorithm  to  remove  redundancies  and  significantly  decrease  the  file  size 
required  to  store  the  voxel  data  cubes.  (Most  AIRIS  voxels  had  values  of  zero.)  As  in 
2007,  only  2D  data  were  available  in  real-time.  The  3D  voxel  data  cubes  were 
retrospectively  reconstructed  from  the  2D  data. 

One  of  the  new  developments  for  the  2008  test  series  was  a  „stack  release"  where  liquid 
simulant  was  piped  to  a  point  high  above  ground  and  then  vaporized.  A  stack  release 
efficiently  supplied  a  constant  stream  of  simulant  for  cloud  formation  at  an  altitude  much 
easier  to  detect  with  standoff  sensors  than  had  been  observed  with  ground-based 
explosive  releases.  Figure  16  shows  the  response  of  the  AIRIS  detectors  (red)  to  a  30  kg 
stack  release  of  TEP  (test  #10T030).  The  format  is  identical  to  that  of  Figure  15,  and  the 
incipient  detection  times  for  AIRIS  are  provided  in  minutes  in  the  upper  right  of  the 
graphs.  In  addition,  the  AIRIS  responses  shown  correspond  to  the  altitude  layer  at  that 
LCD  location  for  which  the  detected  concentration  was  largest  (i.e.,  the  largest  detected 
coincident  AIRIS  response).  As  in  the  earlier  test,  the  results  were  consistent  with  a 
simulant  cloud  dispersing  across  the  test  grid  in  a  non-linear  manner  both  spatially  and 
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Figure  16.  Comparison  of  LCD  (blue)  and  AIRIS  (red)  responses  versus  time  after  release  for  a  30  kg  stack 
release  of  TEP  (test  #10T030).  AIRIS  data  are  the  maximum  concentration  over  all  levels  at  that  LCD 
grid  location.  The  numbers  indicate  the  start  times  for  TEP  detection  for  LCD  (left)  and  AIRIS 
(right). 


temporally.  Further,  the  much  weaker  LCD  responses  (blue)  and  much  later  incipient 
times  (10  to  20  minutes)  were  consistent  with  non-linear  cloud  dispersion  all  the  way 
down  to  ground  level. 

As  in  the  2007  test  series,  weather  data  in  the  2008  test  series  were  acquired  via  the  25 
SPIDAR  Airmar  and  the  five  Dugway  PWIDS  weather  stations.  Weather  data  reporting 
was  more  successful  for  SPIDAR  in  2008  than  in  2007,  with  little  variation  from  the 
PWIDS.  Similar  agreement  as  with  2007  data  was  observed  in  the  temperature,  pressure, 
and  relative  humidity  data,  and  similar  variance  was  observed  in  wind  speeds  and 
directions,  which  certainly  contributed  to  the  non-linear  dispersion  of  the  simulant  cloud. 
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Figure  17.  Temperature  data  in  degrees  Celsius  versus  time  at  heights  of  2  m,  4  m,  8  m,  16  m,  and  32  m  recorded 
at  center  grid  location  on  06/13/2008. 


Figure  18.  Temperature  data  in  degrees  Celsius  versus  time  at  heights  of  2  m,  4  m,  8  m,  16  m,  and  32  m  recorded 
at  center  grid  location  on  06/16/2008. 

New  for  the  2008  test  series  was  the  32  m  tall  mast  at  the  center  grid  location.  The 
mast  was  instrumented  with  temperature  sensors  at  heights  of  2  m,  4  m,  8  m,  16  m,  and 
32  m  above  ground  level  and  provided  data  at  ten  second  intervals  for  all  test  days. 
Figure  17  and  18  display  temperature  data  collected  on  the  center  mast  for  the  days  of 
06/13/2008  and  06/16/2008  in  UTC  time,  respectively.  Local  time  is  UTC  time  minus 
six  hours.  That  is,  midnight  in  the  figure  is  6  pm  local  time.  The  wide  band  on  the  left 
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side  of  the  figures  corresponds  to  night;  the  narrower  band  on  the  right  side  with  day. 
During  the  day  the  temperature  variance  with  respect  to  height  had  a  maximum  of  about 
3  °C  from  ground  level  to  the  top  of  the  mast.  However,  at  night,  when  most  RTVS 
testing  occurred,  the  temperature  variance  was  much  larger  (up  to  10  °C)  and  inverted 
(coolest  at  ground  level).  The  standoff  IR  sensors,  such  as  AIRIS  and  MESH  FTIR, 
depend  on  an  accurate  measurement  of  the  ambient  air  temperature  in  order  to  detect 
interesting  foreground  deviations  (i.e.,  potential  chemical  clouds)  from  ambient 
background.  During  the  RTVS  test  series,  the  standoff  IR  systems  relied  on  temperature 
measurements  made  at  ground  level,  which  have  shown  little  variance  with  location  (see 
Figure  13).  The  demonstrated  variance  with  height  may  have  adversely  impacted  the 
measurements  made  by  the  IR  systems  for  regions  of  the  sky  several  meters  above 
ground  level.  For  instance,  Figure  19  displays  the  temperature  dependence  of  the 
blackbody  radiation  curve  at  temperatures  of  11  °C  and  15  °C  for  infrared  wavelengths. 
The  maximum  variation  of  intensity  with  respect  to  wavelength  occurs  in  the  infrared. 
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Figure  19.  Blackbody  radiation  curves  for  temperatures  of  11  °C  and  15  °C. 
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5.3.  Data  Fusion 


5.3.1  Data  Assimilation 

From  the  perspective  of  data  fusion,  the  RTVS  analysis  presented  several  challenges. 
The  first  of  these  was  that  the  RTVS  sensors  were  disparate  in  modality,  location,  and 
number.  Standoff  and  point-type  sensors  acquired  concentration  and  class  data  at 
different  rates  from  a  number  of  spatial  locations.  A  strategy  was  developed  for  a 
common  spatial  and  temporal  approach  to  the  sensor  data  from  which  a  higher-level 
analysis  on  the  combined  dataset  could  be  performed. 

The  spatial  resolution  of  the  data  at  ground  level  is  shown  in  Figure  20  in  northing  and 
easting  coordinates  for  the  V-Grid  from  the  2007  RTVS  testing.  Here,  the  black  circles 
indicate  the  locations  of  the  LCD  and  ppbRAE  detectors  on  the  test  grid.  The  densely 
hatched  blue  lines  indicate  the  pixels  (4  m  x  4  m)  of  the  AIRIS  standoff  sensors.  As  part 
of  the  tomographic  reconstruction,  the  extent  of  AIRIS  coverage  was  intentionally 
restricted  to  the  test  grid.  The  larger  green  lines  indicate  the  pixels  (50  m  x  50  m)  of  the 
MESH  FTIR  standoff  sensors.  (Processed  FTIR  data  actually  consisted  of  15  m  x  15  m 
pixels,  a  finer  resolution  than  shown.)  Increased  coverage  area  is  a  tradeoff  with  pixel 
resolution,  as  shown  in  Figure  21,  a  zoomed  version  of  Figure  20  that  displays  a  detailed 
view  of  the  respective  standoff  sensor  pixel  areas  surrounding  a  single  point  sensor 
location.  The  spatial  description  of  the  RTVS  sensor  data  for  V-Grid  is  functionally 
identical  to  Target-S  from  2008. 
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Figure  20.  Locations  of  LCD  point  sensors  (black  circles),  AIRIS  (blue  lines)  and 
FTIR  (green  lines)  grids  -  wide-area  view. 
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Figure  21.  Locations  of  LCD  point  sensors  (black  circles),  AIRIS  (blue  lines)  and  FTIR 
(green  lines)  grids  -  zoom  view. 


The  AIRIS  sensors  provided  a  third  spatial  dimension  of  concentration  data  indexed  by 
height  above  the  test  grid.  Tomographic  reconstruction  generated  3D  voxels  of 
dimension  6  m  in  height,  (i.e.,  the  z-direction).  Although  the  data  cube  contained  256 
voxels  in  z,  referred  to  as  z-bins,  the  layout  of  the  source  images  restricted  their  output  to 
bins  128  to  255,  equivalent  to  ground  level  to  a  maximum  height  of  over  half  a  kilometer. 
A  3D  diagram  of  the  AIRIS  voxels  is  shown  in  Figure  22.  The  test  grid  and  LCD 
locations  are  shown  at  ground  level  and  the  grid  lines  correspond  to  the  vertical  layers  of 
AIRIS  voxels.  Together,  ground  level  sensors  and  the  reconstructed  AIRIS  data  provided 
a  three-dimensional  view  of  the  test  grid  that  tracked  the  extent  and  movement  of  the 
simulant  cloud  across  the  grid.  Note  also  that  three-dimensional  data  were  not  available 
for  the  MESH  FTIR  sensors.  Rather,  FTIR  data  were  compressed  to  a  2D  layer  for 
single,  representative  time  slices  and  supplied  as  images  not  suitable  for  computer-based 


z-bin  #128 


Figure  22.  AIRIS  z-bins. 
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analysis. 


A  temporal  description  of  the  RTVS  data  is  supplied  in  Table  2.  The  time  resolution  of 
the  data  is  listed  in  hertz  and  seconds  for  the  various  sensors.  The  LCD  provided  data 
points  at  5-second  intervals.  The  AIRIS  system  processed  data  at  2-second  intervals  in 
real  time  (listed  as  „raw"),  but  the  resolution  of  the  supplied  data  cube  was  reduced  to  10- 
second  intervals  after  tomographic  reconstruction  (listed  as  processed").  For 
comparison,  the  time  resolution  of  the  Dugway  legacy  ppbRAE  point  sensors  was  10- 
second  intervals,  and  for  the  FTIR  standoff  sensors,  30-second  intervals.  Note  that  the 
time-resolved  data  from  the  FTIR  sensors  were  not  available.  Only  2D  images  at 
selected  time-slices  were  published.  The  time  resolution  of  the  weather  data  was  5 
seconds  for  the  LCD  stations  and  10  seconds  for  the  Dugway  PWIDS  stations. 


Table  2.  Data  acquisition  rates. 


System 

Frequency  (Hz) 

Time  (s) 

LCD 

0.2 

5 

AIRIS  (raw) 

0.5 

2 

AIRIS  (processed) 

0.1 

10 

ppbRAE 

0.1 

10 

FTIR  (processed) 

0.03 

30 

LCD  Weather  Stations 

0.2 

5 

PWIDS  Weather  Stations 

0.1 

10 

Temporally,  the  RTVS  datasets  were  registered  to  a  common  time  grid  with  a  10-second 
resolution,  matching  that  of  the  AIRIS  and  ppbRAE  data.  The  LCD  data  were 
interpolated  to  a  10-second  resolution  using  an  industry  standard,  spline-based 
interpolation  technique  provided  by  Matlab"s  „interp"  family  of  functions  [15].  Spline- 
based  interpolation  is  well  suited  for  relatively  well-behaved  data  (e.g.,  continuously 
differentiable),  such  as  the  RTVS  data. 

Spatially,  the  RTVS  datasets  were  analyzed  in  two  different  modes:  columnar  and 
layered.  In  the  columnar  approach,  the  AIRIS  standoff  data  were  compared  to  the  point 
sensor  data  at  each  of  the  25  locations  (see  Figure  7).  For  this  analysis,  the  set  of  AIRIS 
data  voxels  that  formed  the  entire  column  of  data  above  each  location  was  compared  to 
the  LCD  and  ppbRAE  data  at  ground  level  for  that  location.  In  the  layered  approach,  the 
LCD  data  were  compared  to  the  ground  level  layer  of  the  AIRIS  data  and  nearby  layers 
above  ground  level.  For  this  analysis,  a  layer  of  point  sensor  data  with  the  same  spatial 
resolution  as  the  AIRIS  data  were  constructed  from  the  LCD  data  by  interpolating 
spatially  across  the  locations  separately  for  every  point  in  time.  Spline-based 
interpolation  techniques,  as  in  the  temporal  interpolation  described  above,  were 
employed.  Results  from  these  analyses  were  presented  as  movies  constructed  from 
images  of  the  combined  data  at  various  time  steps.  These  results  are  discussed  further 
below. 
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5.3.2  Data  Quality 

The  second  challenge  for  the  RTVS  data  fusion  analysis  was  to  capture  and  leverage  as 
much  information  as  possible  from  the  sensor  algorithms  themselves.  Fundamental  to 
this  effort  was  the  evaluation  of  the  quality  of  the  datasets,  their  consistency  and  their 
reproducibility  in  response  to  varying  levels  of  simulants.  The  development  of  the  RTVS 
sensors  incorporated  sophisticated  methods  for  calibration,  data  acquisition,  and  data 
processing  that  ideally,  were  the  best  available  for  their  respective  instruments. 
Successful  fusion  with  data  obtained  from  multiple  sensing  modalities  (e.g.,  RTVS 
standoff  and  point  sensors)  requires  a  solid  understanding  of  the  resulting  data  quality  in 
order  to  limit  uncertainties  that  may  be  magnified  into  erroneous  information  by  data 
fusion  algorithms  further  up  the  processing  stream. 

Overall,  the  LCD  detectors  and  AIRIS  sensors  were  consistently  responsive  to  the  TEP 
and  MeS  simulants.  The  sensor  responses  were  not  in  general  smooth,  but  were  observed 
to  be  somewhat  choppy  and  intennittent,  which  can  be  seen  in  a  close  examination  of  the 
concentration  profiles  of  Figure  9,  Figure  10,  Figure  12,  Figure  15,  and  Figure  16,  for 
example.  There  was  some  question  as  to  whether  or  not  the  AIRIS  sensors  were 
detecting  much  of  the  simulant  clouds  from  explosive  releases,  but  as  the  AIRIS  response 
was  quite  similar  to  that  of  the  MESH  FTIR  standoff  sensors,  and  both  responded  much 
stronger  to  simulant  from  stack  releases,  it  appears  safe  to  conclude  that  the  lack  of  data 
points  was  mostly  an  artifact  of  the  dissemination  method  and  not  due  to  an  ineffective 
calibration  of  the  standoff  sensors. 

The  effort  put  forth  to  develop  a  rigorous  and  sensible  method  for  calibrating  the  LCD 
detectors  appears  to  have  paid  off  in  reproducible  data  with  a  consistent  baseline  and 
predictable  response  across  the  test  grid  and  simulant  classes.  A  consistent  baseline  was 
observed  to  be  quite  a  problem  with  the  ppbRAE  detectors  whose  response  was 
frequently  shifted  by  amounts  on  the  order  of  the  concentrations  measured.  The  effect 
was  particularly  prominent  in  the  2007  V-Grid  tests.  For  the  2008  Target-S  tests,  the 
response  of  the  ppbRAE  detectors  was  less  than  desired  to  the  TEP  and  MeS  classes  and 
baseline  problems  continued.  Note  that  the  ppbRAE  detectors  responded  well  to  acetic 
acid,  as  shown  in  Figure  23,  though  baseline  problems  are  apparent.  The  LCD  sensors 
were  not  configured  to  detect  acetic  acid. 
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Figure  23.  Response  of  ppbRAE  detectors  to  acetic  acid  in  test  #36AA075.  The  upper  ppbRAE  detector  is  in 
blue,  the  lower  in  green. 


The  temperature  gradient  with  respect  to  height  above  ground,  shown  in  Figure  17  and 
Figure  18,  may  have  affected  the  calibration  and  responses  of  the  AIRIS  and  MESH  IR 
sensors,  which  rely  on  an  accurate  measurement  of  the  background  ambient  air 
temperature  in  order  to  detect  and  measure  foreground  features.  Ambient  temperature 
values  also  appear  in  a  number  of  the  equations  used  in  tomographic  reconstruction. 
However,  a  direct  estimate  of  the  effect  of  an  erroneous  ambient  temperature  is  difficult 
to  determine  without  modeling  the  entire  reconstruction  procedure  using  pure  input  and 
output  signals  of  varying  intensities. 


Some  issues  were  observed  with  the  consistency  of  the  data  obtained  from  all  sensor 
systems.  For  example,  the  response  of  an  LCD  detector  occasionally  appeared  to 
erroneously  drop  to  zero.  These  phenomena  appeared  to  be  a  general  characteristic  of  the 
LCD  sensor  responses  rather  than  a  problem  with  an  individual  malfunctioning  detector. 
Data  dropouts  caused  discontinuities  in  the  concentration  profiles  and  were  problematic 
for  the  temporal  and  spatial  interpolation,  which  was  how  they  were  uncovered. 


35 


A  further  aspect  to  be  considered  with  the  consistency  of  the  data  was  its  variability,  and 
the  implications  of  those  variances  for  false  positives  and  false  negatives.  Here,  false 
positives  are  defined  as  incorrect  indications  of  a  threat  (incorrect  detections),  and  false 
negatives  as  incorrect  indications  of  the  absence  of  a  threat  (incorrect  all-clear  signals). 
As  mentioned  earlier,  sensor  responses  were  generally  not  smooth,  but  often  choppy  with 
a  significant  variance.  Figure  15  shows  the  concentration  profiles  (blue)  from  several 
LCD  detectors,  nearly  all  of  which  exhibit  a  large  variance.  (Note  for  comparison,  the 
LCD  concentration  profile  shown  in  Figure  24  is  considered  to  be  smoothly  varying, 
particularly  after  dropout  correction  (bottom)).  Figure  16  shows  concentration  profiles 
(red)  from  several  individual  AIRIS  voxels  coincident  with  LCD  detectors.  These 
profiles  also  exhibit  a  significant  variance. 


Figure  24.  (Top)  Example  of  LCD  data  (blue  dots)  with  dropouts,  timescale 
grid  (green  lines)  of  10-second  resolution.  (Bottom)  LCD  data  after 
correction  and  interpolation. 
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The  tomographic  reconstruction  methods  employed  by  the  AIRIS  and  MESH  FTIR 
sensor  systems  also  introduced  a  large  number  of  artifacts  into  the  standoff  data.  Figure 
25  shows  the  reconstructed  AIRIS  voxels,  color-coded  by  z-bin  level.  The  regions  of 
detection  show  hard  boundary  edges  in  line  formations  that  are  coincident  with  the 
locations  of  the  AIRIS  sensors.  The  coincidence  is  evidence  that  the  line  features  are 
artifacts  due  to  the  reconstruction,  rather  than  detected  features  of  the  simulant  cloud. 
Gaps  and  additions  along  these  line  features  may  also  be  artifacts  of  the  reconstruction 
algorithms.  Figure  26  shows  four  different  time  slices  of  FTIR  data  from  test  #10TE030. 
The  green  dots  represent  the  locations  of  the  FTIR  sensors  and  the  white  lines  radiating 
from  these  locations  provide  an  indication  of  each  sensor"s  field  of  view  relative  to  the 
reconstruction.  Evidence  for  linear  features,  gaps,  and  other  areas  of  potential  added 
material,  is  easier  to  discern  in  these  images.  A  likely  explanation  is  that  one  of  the 
sensors  detected  some  level  of  simulant  in  a  particular  pixel  in  its  2D  view,  but  the 
reconstruction  algorithms  were  not  able  to  determine  accurately  where  that  simulant 
measurement  was  located  with  respect  to  the  other  sensors.  Therefore,  uncertainty 
remained  as  to  how  far  away  from  the  sensor  the  simulant  was  detected.  Compounding 
these  uncertainties  across  all  of  the  2D  data  leads  to  observed  artifacts  in  the 
reconstructed  3D  data.  Note  that  similar  artifacts  were  observed  with  both  the  AIRIS  and 
MESH  FTIR  data  even  though  different  reconstruction  algorithms  were  employed. 
Tomographic  reconstruction  in  three  dimensions  is  by  nature  an  ill-posed  inversion 
problem  that  is  only  made  more  difficult  when  variances  are  present  in  the  2D  source 
data. 

Data  variance  and  artifacts  need  be  understood  and  quantified  for  effective  data  fusion. 
In  standard  signal  processing  analysis,  the  variance  of  a  signal  is  considered  a  good 
indication  of  the  inherent  uncertainty  of  that  signal.  That  is,  a  large  variance  implies  a 
large  uncertainty  in  any  sample  data  point.  With  respect  to  the  RTVS  sensors,  however, 
the  impact  of  several  factors  on  data  variance  must  also  be  evaluated.  These  factors 
include  environmental  conditions  at  the  test  site,  the  dissemination  methods  used  to 
generate  the  simulant  clouds,  and  the  interactions  of  these  with  characteristics  of  the 
sensors  themselves.  Columnar  and  planar  analyses  were  used  to  investigate  these  factors. 

As  mentioned  earlier,  a  columnar  analysis  was  performed  by  examining  the  sensor  data  at 
the  locations  of  the  25  LCD  detectors.  At  the  bottom  of  the  data  column  (ground  level), 
there  were  data  from  the  LCD  detector  and  the  lowest  AIRIS  voxel  in  the  column.  The 
remainder  of  the  data  column  consisted  of  the  other  127  AIRIS  voxels  coincident  with  the 
LCD  detector  but  increasing  in  height  z  to  the  top  of  the  AIRIS  data  cube.  It  was 
expected  that  a  simulant  cloud  passing  through  such  a  data  column  would  exhibit 
connectivity  between  voxels  adjacent  in  height,  and  if  coincident  with  ground  level,  the 
LCD  detector  as  well.  Little  evidence  for  connectivity  or  coincidence  was  observed  in 
the  RTVS  data. 
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Figure  25.  AIRIS  voxels  from  tomographic  reconstruction  at  ground  level. 


t. 


Tables  3,  4,  5,  and  6  are  examples  of  the  columnar  analysis  for  the  2008  Target-S  tests. 
Table  3  was  an  explosive  release  of  150  kg  TEP  simulant  (test  #05TE150),  and  Table  4 
was  an  explosive  release  of  30  kg  of  TEP  (test  #06TE030).  Table  5  was  an  explosive 
release  of  150  kg  of  TEP  (test  #09TE150),  and  Table  6  was  a  stack  release  of  30  kg  of 
TEP  (test  #10TE030).  In  these  tables,  column  data  is  turned  on  its  side.  That  is,  the 
tables  list  by  row  the  concentration  data  in  lines  (really,  columns)  rising  up  vertically 
from  the  locations  of  the  LCD  detectors.  (The  locations  of  the  LCD  detectors  according 
to  Figure  7  are  provided  in  the  first  column  of  the  table.)  The  concentration  values  are  in 
units  of  mg/m3  and  represent  a  summation  over  all  time  steps  for  which  data  were 
available  for  that  test.  The  number  of  time  steps  is  constant  for  all  AIRIS  data  and  is 
provided  in  the  table  header  row.  The  first  table  column  of  AIRIS  data  contains  the  sums 
over  all  voxels  (z-bins)  for  each  data  column  (i.e.,  row  in  the  table).  The  second  AIRIS 
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Figure  26.  Four  different  time  slices  of  FTIR  data  from  test  #10TE030. 


data  column,  z-bin  #128,  contains  the  concentrations  of  the  ground  level  voxels.  Each 
subsequent  z-bin  column  increases  in  height  by  6  m.  Total  values  of  the  concentrations 
for  all  locations  by  height  are  provided  in  the  bottom  row. 

Examination  of  table  data  shows  that  there  was  little  coincidence  between  the  LCD 
detectors  and  AIRIS  sensors.  For  example,  the  explosive  releases  in  Tables  3  and  4  have 
very  little  data  from  the  AIRIS  sensors  for  any  height.  Even  when  those  data  are 
consistent  between  sensors,  as  at  location  SE  in  Table  3,  the  connectivity  is  awry.  Here, 
the  largest  concentrations  seen  by  AIRIS  are  in  the  two  z-bins  (#129  and  #130)  above 
ground  level  (z-bin  #128)  while  the  ground  level  concentration  (z-bin  #128)  disagrees 
significantly  with  that  measured  by  the  LCD  detector.  A  similar  pattern  is  seen  in  Table 
4.  Better  coincidence  between  the  sensors  is  seen  in  Table  5.  With  the  stack  release  of 
Table  6,  more  simulant  was  released  much  higher  above  ground  and  consequently  the 
AIRIS  data  track  the  simulant  cloud  much  more  consistently.  However,  there  is  still  a 
large  disagreement  between  the  LCD  and  AIRIS  systems  at  ground  level,  particularly  in 
Table  6.  These  relationships  between  the  sensors  persist  even  if  the  concentration  values 
are  rounded  to  the  nearest  order  of  magnitude  in  order  to  roughly  approximate  the 
difference  in  integration  times,  a  factor  which  was  not  considered  in  these  tables. 
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Table  3.  Columnar  analysis  for  Test  #05TE150 


LCD  Concentraions 

AIRIS  Concentrations  at  Various  Z-Bins 

(98  timesteps ) 

Location 

Values 

Timesteps 

All  Z  Bins 

Z#128 

Z#129 

Z#130 

Z#131 

Z#132 

Z#133 

NW  Comer 

1526.63 

1431 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

N  Edge 

280.29 

1647 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE  Corner 

79.44 

1549 

0.47 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

WEdge 

43.41 

1667 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Center 

12.03 

1593 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

E  Edge 

240.46 

1650 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW  Corner 

2.53 

1690 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

S  Edge 

0.00 

0 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SE  Corner 

58.34 

1596 

23.34 

0.48 

10.29 

12.57 

0.00 

0.00 

0.00 

NW1 

1213.63 

1630 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW2 

0.00 

0 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW3 

745.74 

1599 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW4 

296.11 

1537 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE1 

96.34 

1622 

1.82 

0.00 

0.00 

0.47 

1.34 

0.00 

0.00 

NE2 

1028.40 

1548 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE3 

11.69 

1668 

1.32 

0.00 

0.00 

1.32 

0.00 

0.00 

0.00 

NE4 

706.56 

1442 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW1 

2.57 

1696 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW2 

0.00 

0 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW3 

22.28 

1663 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW4 

32.29 

1631 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SE1 

1.06 

1693 

0.11 

0.00 

0.00 

0.11 

0.00 

0.00 

0.00 

SE2 

0.00 

0 

3.39 

0.00 

3.08 

0.31 

0.00 

0.00 

0.00 

SE3 

0.00 

0 

0.04 

0.00 

0.02 

0.02 

0.00 

0.00 

0.00 

SE4 

77.21 

1687 

3.57 

0.00 

3.56 

0.00 

0.01 

0.00 

0.00 

Totals 

6477.02 

34.06 

0.48 

16.96 

14.80 

1.35 

0.00 

0.00 
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Table  4.  Columnar  analysis  for  Test  #06TE030 


LCD  Concentrations 

AIRIS  Concentrations  at  Various  Z-Bins 

( 157  timesteps ) 

Location 

Values 

Timesteps 

All  Z  Bins 

Z#128 

Z#129 

Z#130 

Z#131 

Z#132 

Z#133 

Z#134 

Z#135 

Z#136 

Z#137 

NWC 

0.00 

0 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

N  Edge 

35.18 

83 

1.26 

1.24 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NEC 

59.69 

143 

11.42 

11.19 

0.23 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

WEdge 

0.00 

0 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Center 

0.00 

0 

0.57 

0.00 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.56 

0.00 

E  Edge 

299.91 

181 

26.27 

11.93 

9.16 

5.18 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

swc 

1.39 

23 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

S  Edge 

1.79 

29 

0.03 

0.00 

0.00 

0.03 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SEC 

0.00 

0 

7.18 

1.31 

0.61 

0.69 

4.57 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

NW1 

8.82 

19 

0.02 

0.00 

0.00 

0.02 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW2 

26.16 

80 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW3 

42.05 

116 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW4 

190.29 

204 

0.58 

0.04 

0.00 

0.54 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE1 

6.75 

111 

0.12 

0.11 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE2 

4.34 

125 

0.38 

0.34 

0.04 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE3 

1.64 

85 

3.00 

2.10 

0.00 

0.89 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE4 

8.17 

43 

14.82 

0.58 

14.22 

0.03 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW1 

13.41 

213 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW2 

0.00 

0 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW3 

19.27 

67 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW4 

172.91 

215 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SE1 

0.00 

0 

0.06 

0.01 

0.00 

0.03 

0.01 

0.00 

0.00 

0.00 

0.00 

0.01 

0.00 

SE2 

0.00 

0 

3.01 

1.59 

0.40 

1.00 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

SE3 

0.00 

0 

1.58 

0.78 

0.60 

0.19 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SE4 

2.29 

88 

37.91 

15.48 

14.61 

4.38 

3.44 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Totals 

894.08 

108.24 

46.70 

39.89 

12.98 

8.04 

0.02 

0.00 

0.00 

0.00 

0.58 

0.02 
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Table  5.  Columnar  analysis  for  Test  #09TE150 


LCD  Concentrations 

AIRIS  Concentrations  at  Various  Z-Bins  i 

( 107  timesteps) 

Location 

Values 

Timesteps 

All  Z  Bins 

Z#128 

Z#129 

Z#130 

Z#13l 

Z#132 

Z#133 

Z#134 

Z#135 

Z#136 

Z#137 

NWC 

52.63 

1081 

36.44 

0.00 

0.00 

0.00 

1.96 

24.98 

9.11 

0.39 

0.00 

0.00 

0.00 

N  Edge 

66.00 

1158 

0.02 

0.00 

0.00 

0.00 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

NEC 

11.65 

1226 

0.07 

0.00 

0.00 

0.00 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.04 

WEdge 

250.83 

970 

25.08 

3.96 

3.08 

13.36 

4.69 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

Center 

55.80 

1199 

47.04 

0.01 

1.44 

0.00 

0.00 

0.00 

0.02 

0.09 

0.00 

8.66 

11.19 

E  Edge 

5.46 

1260 

0.32 

0.00 

0.00 

0.00 

0.00 

0.00 

0.31 

0.01 

0.00 

0.00 

0.00 

swc 

13.82 

992 

0.06 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.04 

S  Edge 

12.32 

1114 

25.20 

0.07 

2.95 

3.64 

13.89 

2.71 

1.95 

0.00 

0.00 

0.00 

0.00 

SEC 

1102.39 

1200 

80.90 

0.26 

7.70 

15.38 

28.94 

28.60 

0.01 

0.00 

0.00 

0.00 

0.00 

NW1 

0.00 

0 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NW2 

0.00 

0 

0.07 

0.00 

0.00 

0.00 

0.00 

0.00 

0.07 

0.00 

0.00 

0.00 

0.00 

NW3 

41.80 

1131 

4.62 

0.02 

1.74 

2.78 

0.00 

0.00 

0.00 

0.06 

0.00 

0.00 

0.00 

NW4 

23.59 

1267 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE1 

118.66 

850 

0.05 

0.02 

0.00 

0.00 

0.00 

0.00 

0.00 

0.01 

0.00 

0.00 

0.02 

NE2 

4.95 

1200 

0.03 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.02 

NE3 

132.07 

1030 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

NE4 

7.94 

1264 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW1 

223.09 

1202 

27.85 

0.00 

17.01 

8.48 

2.32 

0.00 

0.00 

0.00 

0.03 

0.00 

0.01 

SW2 

67.18 

1118 

24.81 

5.84 

5.48 

6.68 

6.77 

0.00 

0.00 

0.00 

0.02 

0.00 

0.01 

SW3 

438.86 

1192 

15.97 

0.00 

5.64 

4.69 

5.64 

0.00 

0.00 

0.00 

0.00 

0.00 

0.00 

SW4 

92.88 

1081 

88.65 

11.99 

17.68 

38.44 

20.01 

0.53 

0.00 

0.00 

0.00 

0.00 

0.00 

SE1 

0.00 

0 

18.22 

0.03 

6.11 

1.03 

0.43 

3.13 

6.08 

1.42 

0.00 

0.00 

0.00 

SE2 

0.00 

0 

2.29 

0.16 

0.00 

0.00 

0.09 

1.62 

0.23 

0.17 

0.00 

0.00 

0.00 

SE3 

0.00 

0 

76.03 

22.50 

14.15 

21.01 

7.79 

9.26 

1.31 

0.02 

0.00 

0.00 

0.00 

SE4 

192.70 

1234 

51.05 

0.00 

0.01 

0.00 

0.00 

22.55 

16.18 

12.32 

0.00 

0.00 

0.00 

Totals 

2914.62 

524.79 

44.87 

82.99 

115.48 

92.53 

93.42 

35.27 

14.48 

0.07 

8.67 

11.33 
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Table  6.  Columnar  analysis  for  Test  #10TE030 


LCD  Concentrations 

AIRIS  Concentrations  at  Various  Z-Bins 

(193  timesteps) 

Location 

Values 

Timesteps 

All  Z  Bins 

Z#128 

Z#129 

Z#130 

Z#131 

Z#132 

Z#133 

Z#134 

Z#135 

Z#136 

Z#137 

NWC 

31.44 

408 

76.35 

0.00 

0.00 

1.13 

0.00 

0.02 

0.04 

0.06 

6.89 

37.88 

19.68 

N  Edge 

0.00 

0 

60.06 

0.00 

0.00 

45.24 

1.14 

1.35 

0.00 

0.00 

1.47 

5.57 

2.37 

NEC 

0.14 

407 

96.94 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.00 

0.69 

37.46 

34.53 

WEdge 

18.98 

597 

12.85 

0.00 

0.00 

0.07 

0.00 

0.04 

0.06 

0.16 

0.00 

0.65 

3.01 

Centrr 

8.02 

503 

299.17 

0.00 

0.00 

1.77 

0.25 

0.07 

0.00 

2.02 

17.17 

170.05 

94.15 

E  Edge 

0.16 

408 

1.56 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.05 

0.33 

0.09 

0.02 

swc 

0.90 

454 

21.55 

0.00 

0.00 

0.01 

0.00 

0.00 

0.00 

0.02 

1.80 

0.35 

6.25 

S  Edge 

3.02 

409 

40.44 

0.00 

0.00 

0.09 

0.05 

0.00 

0.03 

16.59 

20.02 

2.54 

0.95 

SEC 

167.11 

696 

259.84 

0.00 

11.84 

123.25 

37.36 

0.15 

0.00 

0.29 

9.06 

54.90 

21.38 

NW1 

0.54 

536 

31.10 

0.00 

0.00 

0.07 

0.00 

0.02 

0.04 

0.08 

0.17 

4.03 

5.17 

NW2 

0.00 

0 

23.47 

0.00 

0.00 

3.81 

0.04 

0.00 

0.01 

0.02 

2.49 

9.03 

7.14 

NW3 

0.00 

0 

80.49 

0.00 

0.00 

0.32 

7.30 

0.00 

0.03 

0.01 

2.69 

16.50 

37.59 

NW4 

1.30 

364 

70.97 

0.00 

0.01 

1.01 

19.45 

0.06 

0.02 

2.26 

16.09 

14.18 

11.13 

NE1 

7.39 

412 

56.56 

0.00 

0.00 

0.62 

1.25 

0.14 

0.00 

0.01 

23.74 

4.57 

5.87 

NE2 

14.48 

433 

91.81 

0.00 

0.00 

0.01 

0.53 

0.00 

0.00 

5.80 

10.87 

23.97 

22.99 

NE3 

6.67 

421 

19.77 

0.00 

0.00 

0.11 

1.92 

0.15 

0.00 

0.03 

0.03 

0.80 

14.24 

NE4 

0.00 

0 

25.65 

0.00 

0.01 

0.00 

0.00 

0.00 

0.00 

0.32 

0.12 

1.56 

19.15 

SW1 

2.97 

471 

425.36 

0.00 

0.00 

0.01 

0.00 

0.02 

0.98 

44.19 

161.68 

135.91 

30.94 

SW2 

13.18 

588 

523.27 

0.00 

0.00 

0.09 

0.01 

0.02 

0.04 

7.03 

277.02 

120.66 

73.97 

SW3 

10.24 

561 

10.37 

0.00 

0.00 

0.01 

0.00 

0.05 

0.06 

0.09 

4.46 

1.49 

4.17 

SW4 

88.33 

698 

44.21 

0.00 

0.00 

0.15 

0.04 

0.00 

0.03 

1.04 

4.57 

1.00 

36.71 

SE1 

3.98 

411 

15.42 

0.00 

0.00 

0.42 

0.31 

0.08 

0.00 

6.48 

6.72 

0.48 

0.10 

SE2 

0.00 

0 

4.42 

0.00 

0.00 

0.55 

1.45 

0.15 

0.00 

0.00 

0.28 

0.06 

0.22 

SE3 

0.00 

0 

43.51 

0.00 

0.00 

2.71 

9.42 

4.86 

0.00 

11.71 

11.45 

2.69 

0.12 

SE4 

1.24 

380 

10.33 

0.00 

0.01 

0.06 

2.47 

6.34 

0.00 

0.20 

0.08 

0.11 

0.06 

Totals 

380.07 

2345.48 

0.00 

11.90 

181.51 

83.00 

13.52 

1.34 

98.46 

579.91 

646.53 

451.91 
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The  discrepancy  of  sensors  at  ground  level  could  be  further  investigated  with  the  Dugway 
ppbRAE  detectors.  These  detectors  were  collocated  with  the  LCD  systems  to  within  a 
distance  of  a  few  meters.  At  each  location,  one  ppbRAE  detector  was  placed  1  m  above 
ground,  the  same  height  as  the  LCD  detectors,  while  the  other  was  placed  at  a  height  of  6 
m,  which  placed  it  on  boundary  between  the  first  and  second  AIRIS  voxels  (z-bin  #128 
and  z-bin  #129).  Figure  27  shows  the  response  of  the  ppbRAE  detectors  during  test 
#05TE150,  the  same  test  as  that  of  Table  3.  In  the  figure,  the  response  of  the  upper 
detector  is  shown  in  blue,  the  lower  in  green.  With  the  exception  of  the  NW  corner 
location  (NWC),  the  upper  and  lower  ppbRAE  detectors  exhibited  no  coincident 
detections  with  collocated  detectors  at  different  heights,  which  confirms  the  results  for 
Tables  3  -  6  for  collocated  LCD  and  AIRIS  data.  Further,  the  ppbRAE  detections  were 
generally  not  consistent  with  those  of  Table  3.  These  results  were  typical  for  the  ppbRAE 
data;  see,  for  example,  Figure  23. 
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Figure  27.  Response  of  ppbRAE  detectors  to  triethyl  phosphate  in  test  #05TE150.  The  upper  ppbRAE  detector 
is  in  blue,  the  lower  in  green. 
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Planar  analysis  provided  further  insight  into  the  impact  of  the  method  of  dissemination 
for  data  quality.  For  the  planar  analysis,  a  two-dimensional  interpolation  algorithm  [15] 
was  used  to  interpolate  the  LCD  data  onto  a  grid  of  pixels  matching  the  AIRIS  voxels  at 
the  ground-level  plane  of  the  test  grid.  Interpolation  of  the  LCD  data  was  not  ideal,  as 
the  data  points  were  sparsely  distributed  across  the  grid  with  distances  of  200  m  to  300  m 
between  data  points.  The  effect  of  the  2D  interpolation  was  to  distribute  the  point 
detections  across  a  larger  area  surrounding  each  LCD  location  and  show  links  between 
nearest  neighbor  detectors  that  could  better  delineate  the  extent  of  the  simulant  cloud. 
Because  of  the  sparsity  of  the  LCD  data,  due  partly  to  few  detections  and  partly  to  signal 
variance,  the  movement  of  the  simulant  cloud  across  the  test  grid  was  even  less  clear  than 
with  the  standoff  data.  Further,  there  was  a  little  agreement  between  the  ground-level 
AIRIS  data  and  the  interpolated  LCD  data,  though  this  was  expected  given  the  results  of 
the  columnar  analysis.  The  interpolated  LCD  data  did  illuminate  some  characteristics  of 
explosive  and  stack  dissemination  methods.  The  data  indicated  that  the  LCD  detectors 
were  seeing  simulant  that  was  coming  off  of  the  ground,  which  had  been  suspected 
earlier.  Ground-reflected  simulant  was  likely  due  to  uneven  globular  dissemination  and 
evaporation  of  simulant  during  an  explosive  release.  Simulant  material  being  reflected 
off  the  ground  can  also  explain  the  lack  of  concentration  detected  by  the  AIRIS  sensors 
for  explosive  releases. 

Finally,  in  addition  to  the  temperature  effects  discussed  earlier,  variations  in  the  wind 
directions  and  magnitudes  across  the  test  grid  were  an  important  factor  affecting  how  the 
simulant  cloud  moved  across  the  grid.  Measurements  of  wind  speeds  and  directions  were 
compiled  from  the  Airmar  weather  stations  collocated  with  the  LCD  detectors.  These 
results  were  compiled  into  a  number  of  movies  that  simultaneously  showed  the  winds 
from  each  station.  Compilation  movies  revealed  that  even  when  the  winds  were  flowing 
steadily  in  one  direction  over  the  test  grid  there  was  significant  individual  variation  from 
this  flow  over  short  time  scales  at  every  location.  Examination  of  wind  data  from  the 
PWIDS  stations,  which  continuously  gathered  data,  showed  that  wind  variations  were 
larger  during  the  day  than  the  night.  Night  testing  thus  mitigated  the  wind  effect  on  the 
testing,  but  the  winds  remained  a  significant  source  for  variation  between  point  sensors  at 
the  ground  level  and  for  regions  monitored  by  the  standoff  sensors. 

Thus,  environmental  factors  and  the  dissemination  methods  both  had  significant  effects 
on  data  quality.  Wind  variations  further  aggravated  uneven,  globular  dissemination  and 
evaporation  of  simulant  from  explosive  releases,  and  disrupted  the  generation  of  a 
smooth  plume  of  simulant  from  stack  releases.  It  is  likely  that  the  variations  observed  in 
the  LCD  data  are  due  largely  to  these  two  factors.  It  is  also  likely  that  these  factors 
contributed  to  the  artifact  problem  observed  in  the  standoff  sensor  data. 

5.3.3  Improving  Data  Quality 

A  number  of  techniques  may  be  employed  to  improve  the  quality  of  the  RTVS  data.  For 
instance,  the  LCD  dropouts  are  easy  to  identify  and  repair  retrospectively  and  were 
corrected  as  part  of  the  data  fusion  analysis.  A  dropout  was  identified  as  a  zero-valued 
data  point  for  which  the  preceding  and  subsequent  data  points  were  above  a  small,  non¬ 
zero  threshold.  The  zero-valued  point  was  replaced  with  a  value  interpolated  from 
several  nearest  neighbor  data  points.  Figure  24  (top)  shows  an  example  of  LCD  data 
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(blue  dots)  with  two  dropouts.  The  10-second  time  intervals  are  shown  in  the  green 
vertical  lines.  Figure  24  (bottom)  displays  the  corrected  and  interpolated  LCD  data.  The 
correction  was  perfonned  simultaneously  with  interpolation  of  the  LCD  data  to  the  10- 
second  time  resolution. 

In  real  time,  dropouts  are  more  challenging  to  identify  and  correct  because  of  the 
difficulty  in  distinguishing  a  dropout  from  a  legitimate,  zero-valued  data  point  without 
knowledge  of  the  subsequent  (i.e.,  future)  data  values.  Therefore,  a  real-time  correction 
necessarily  requires  interpolating  over  a  buffer  of  the  most  recent  data  points  and 
consequently  delays  further  upstream  processing  of  the  data  by  several  seconds.  For 
example,  a  buffer  containing  the  ten  most  recent  LCD  data  points  would  incur  a  delay  of 
20  seconds  for  situational  awareness,  given  a  2-second  data  acquisition  rate. 

Similarly,  the  variance  observed  in  the  LCD  concentration  profiles  is  easily  rectified  with 
the  use  of  a  moving  average  filter  or  interpolation  algorithm.  The  bias-variance  tradeoff 
is  a  significant  challenge  when  reducing  variance;  care  needs  to  be  taken  not  to  bias  the 
data  by  over  smoothing  with  a  filter.  Implementing  a  filter  for  real-time  processing  is 
also  likely  to  incur  a  buffer  delay  for  situational  awareness. 

Decreasing  the  variance  observed  in  the  AIRIS  voxels  is  a  more  difficult  problem.  An 
averaging  or  interpolating  filter,  in  time,  space,  or  both  simultaneously,  may  lessen  the 
number  of  artifacts,  improve  the  smoothness,  and  reduce  gaps  in  the  detected  cloud,  but 
will  almost  surely  aggravate  the  intermittent  data  points  appearing  sporadically  in  the 
data  cube  by  enhancing  both  the  spatial  and  temporal  extents  of  these  artifacts.  Rather 
than  averaging,  an  approach  that  favors  adjacent  (nearest  neighbor)  data  points  in  space 
and  persistent  data  points  in  time  would  be  more  effective  at  reducing  intermittent  data 
points  and  reducing  gaps. 

Another  approach  to  decreasing  the  variance  in  AIRIS  data  is  to  consider  the  source  of 
the  artifacts,  the  tomographic  reconstruction  algorithms.  For  instance,  concentrations 
detected  in  the  2D  pixel  data  are  mapped  to  sets  of  voxels  in  the  3D  reconstructed  data. 
If  the  mapped  data  cannot  be  localized  to  an  individual  voxel,  then  its  concentration 
should  be  divided  equally  among  all  possible  voxels  that  form  solutions  to  the 
tomographic  equations  for  this  2D  data.  A  Bayesian  approach  could  refine  this 
equipartition  of  the  concentration  based  on  adjacency  and  persistence  requirements. 
Uncertainties  could  be  estimated  from  the  contributions  and  partitions  of  multiple  pixels. 
Such  Bayesian  values  and  uncertainties  are  well  suited  to  pattern  recognition  and  signal 
processing  algorithms. 

Binning  is  another  approach  appropriate  for  mitigating  the  effects  of  the  widely  varying 
concentrations  seen  in  the  AIRIS  data.  For  example,  reducing  the  concentrations  to  one 
of  four  possible  values  (e.g.,  none,  low,  medium,  high)  makes  it  much  easier  to  identify 
which  voxels  are  most  significant  for  situational  awareness.  The  boundaries  of  the  four 
bins  can  be  suitably  chosen  from  a  histogram  of  the  concentrations  observed  in  the  AIRIS 
data  over  a  variety  of  tests.  Further,  binning  does  not  reduce  the  effectiveness  of  the 
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aforementioned  variance  reducing  approaches  and  may  make  their  implementation  less 
difficult. 


In  all  of  these  approaches,  the  difference  between  a  raw  data  value  and  one  that  has  been 
filtered  or  modified  may  serve  as  a  measure  of  the  potential  uncertainty  in  that  data  value. 
Such  uncertainty  estimates  can  be  very  effective  for  multisensor  data  fusion. 

5.3.4  Data  Fusion 

The  discussion  of  sources  of  variance  in  the  datasets  leads  naturally  to  the  development 
of  a  frame  of  discernment  appropriate  for  RTVS  data  fusion.  First,  the  detections  of  TEP 
and  MeS  simulants  by  the  LCD  point  sensors  were  likely  to  be  correct  detections  with 
very  few  false  positives.  Values  for  the  detected  concentrations  of  simulants  were  also 
reasonable.  The  absence  of  detections  by  an  LCD  point  sensor,  however,  was  not  a  clear 
indication  of  the  absence  of  simulant  when  simulant  had  been  recently  detected  by  the 
sensor,  or  detected  by  an  adjacent  sensor.  Varying  wind  conditions  and  data  dropouts 
were  both  significant  sources  of  false  negatives  for  the  LCD  sensors. 

For  the  AIRIS  data,  detections  of  simulants  were  likely  to  be  correct  indications  that 
simulants  were  present  somewhere  in  the  field  of  view  of  the  sensors.  The  precise 
location  was  uncertain  due  to  tomographic  reconstruction  with  varying  data  sources,  as 
were  the  detected  concentration  values.  The  absence  of  detections  was  only  likely  to  be 
correct  in  the  AIRIS  data  if  no  detections  were  observed  anywhere  in  the  sensors"  fields 
of  view.  Thus,  the  AIRIS  sensors  were  effective  at  determining  whether  a  simulant  was 
present,  but  not  as  effective  at  detennining  where  the  simulant  was  located  at  any  given 
time. 

The  coverage  areas  of  the  LCD  and  AIRIS  sensors  are  effectively  disparate  and  do  not 
overlap.  Though  the  ground-level  boundary  of  the  AIRIS  coverage  area  does  include  the 
point  sensors,  boundary  effects  in  the  tomographic  reconstruction  appear  to  limit  the 
effectiveness  of  AIRIS  at  ground  level.  The  lack  of  overlapping  coverage  from  the  LCD 
and  AIRIS  sensors  limits  the  perfonnance  gains  possible  with  multisensor  data  fusion. 

Together,  the  LCD  and  AIRIS  sensors  form  a  frame  of  discernment  in  which  detections 
of  simulants  are  clear  indications  of  a  tangible  chemical  threat,  and  the  absence  of 
detections  is  an  accurate  all-clear  signal  on  the  test  grid.  However,  the  combined  sensors 
are  not  able  to  effectively  distinguish  local  regions  of  threat  from  regions  of  all  clear  on 
the  test  grid  when  a  simulant  has  been  released. 

This  frame  of  discernment  has  implications  for  the  selection  of  suitable  data  fusion 
algorithms.  The  presence  of  significant  numbers  of  false  negatives  means  that  a 
Dempster-Shafer  approach  would  be  the  most  effective  strategy  for  RTVS  data  fusion,  as 
this  approach  treats  all  types  of  information  as  adding  to  the  total  situational  awareness, 
but  remains  flexible  when  the  absence  of  infonnation  is  uncertain  or  not  informative  for 
situational  awareness.  In  addition,  heuristic  methods  would  be  effective  for 
implementing  the  algorithms  discussed  for  improving  RTVS  data  quality.  Heuristic  rules 
are  also  likely  to  be  useful  for  RTVS  data  fusion  as  well. 
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As  for  the  other  data  fusion  methods  discussed,  a  Bayesian  approach  treats  the  absence  of 
information  with  the  same  significance  for  situational  awareness  as  the  presence  of 
information,  and  is  therefore  not  well  suited  for  RTVS.  False  negatives  also  greatly 
restrict  the  selection  of  potential  pattern  recognition  approaches  for  RTVS  data  fusion,  as 
most  algorithms  treat  information  in  the  same  manner  as  the  Bayesian  approach.  Pattern 
recognition  may  be  effective  at  improving  the  number  and  types  of  artifacts  observed  in 
the  standoff  sensor  data. 

6.  CONCLUSIONS 

•  Environmental  factors  significantly  affect  the  testing  at  Dugway  Proving 
Grounds.  Winds  induced  significant  variance  in  the  RTVS  data.  Testing  at  night 
when  winds  were  less  variable  mitigated  wind  effects.  An  inverted  temperature 
gradient  with  respect  to  height  may  have  affected  the  calibration  and  sensing 
capabilities  of  the  standoff  sensors.  Testing  at  night  increased  the  effect,  as  the 
temperature  gradient  was  positive  and  less  steep  during  the  day. 

•  A  large  variation  in  the  RTVS  data  was  observed  with  respect  to  the 
dissemination  method.  Explosive  dissemination  generated  an  uneven,  globular 
release  of  simulant  with  unpredictable  vaporization.  Dissemination  by  stack 
release  produced  a  more  even  distribution  of  vaporization  simulant,  but  was  not 
consistent  with  the  expected  chemical  threat  scenario. 

•  The  effective  calibration  and  baselines  of  the  LCD  point  sensors  make  them  a 
better  choice  than  the  Dugway  ppbRAE  point  sensors  for  RTVS. 

•  The  three-dimensional  AIRIS  data  cube  with  individual  voxels  covering  the  test 
grid  to  a  great  height  is  an  extremely  flexible  format  for  data  fusion.  The 
problems  observed  with  the  tomographic  reconstruction  will  lessen  with  further 
experience  and  computational  power. 

•  Overall,  the  AIRIS  sensors  provided  cloud-tracking  capabilities  that  were  similar 
to  the  Dugway  MESH  FTIR  sensors,  but  were  at  three-times  better  resolution. 
Cloud  tracking  with  either  AIRIS  or  MESH  was  a  vast  improvement  over  tracking 
with  the  ORION  IR  cameras,  which  typically  lost  track  of  the  cloud  within  two 
minutes  of  its  dissemination. 

•  The  coverage  areas  of  the  LCD  and  AIRIS  sensors  are  effectively  disparate  and 
do  not  overlap,  which  limits  the  perfonnance  gains  possible  with  multisensor  data 
fusion. 

•  Together,  the  LCD  and  AIRIS  sensors  provide  situational  awareness  in  which 
detections  of  simulants  are  clear  indications  of  a  tangible  chemical  threat,  and  the 
absence  of  detections  is  an  accurate  all-clear  signal  for  the  test  grid  as  a  whole. 
The  combined  sensors  are  not  able  to  effectively  distinguish  local  regions  of 
threat  from  regions  of  all  clear  on  the  test  grid  when  a  simulant  has  been  released. 

•  The  presence  of  significant  numbers  of  false  negatives  means  that  a  Dempster- 
Shafer  approach  is  the  most  effective  strategy  for  RTVS  data  fusion.  Dempster- 
Shafer  treats  all  types  of  information  as  adding  to  the  total  situational  awareness, 
but  remains  flexible  when  the  absence  of  information  is  uncertain  or  not 
informative  for  situational  awareness. 
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•  Heuristic  methods  would  be  effective  for  implementing  the  algorithms  discussed 
for  improving  RTVS  data  quality.  Heuristic  rules  are  also  likely  to  be  useful  for 
RTVS  data  fusion. 

•  Real-time  situational  awareness  is  possible  from  fused  LCD  and  AIRIS  data,  but 
the  rate  is  limited  to  at  best  one  update  per  minute  covering  the  whole  test  grid. 
The  performance  will  improve  with  increased  computational  power  and  better 
reconstruction  algorithms. 
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APPENDIX  A 


RTVS  dissemination  logs  for  the  2007  (Table  Al)  and  2008  (Table  A2)  test  series. 


A-l 


Table  Al.  RTVS07  Dissemination  Log  Data 


Trial 

Name 

Date 

(Sept 

2007) 

Referee 

Start 

Time 

Release 

Time 

Trial 

End 

Time 

Dissemination 

Area3 

Dissemination  Points'3 

Amount  of  TEPb 
Disseminated 
(kg) 

01T010 

17 

13:15:00 

13:30:00 

13;43:00 

NNW 

14,15,18,19 

10 

01T010 

18 

07:30:00 

07:45:00 

08:03:00 

NNW 

14,15,18,19 

10 

02T010 

18 

14:14:00 

14:30:00 

14:36:00 

SSE 

14,15,18,19,22,23 

30 

03T030 

19 

07:45:00 

07:57:00 

08:25:00 

SSW 

El  3 

30 

04T030 

19 

08:55:00 

09:08:00 

09:34:00 

SSW 

El  3 

50 

05T030 

19 

10:03:00 

10:18:00 

11:02:00 

SSW 

El  3 

30 

06T100 

19 

11:02:00 

11:45:00 

12:20:00 

SSW 

El  3 

100 

07T050 

20 

10:58:00 

11:18:00 

11:39:00 

SSE 

13,14,15,16,17,18,19,20,22,23 

50 

08T050 

20 

12:27:00 

12:41:00 

12:51:00 

SSE 

13,14,15,16,17,18,19,20,22,23 

50 

09T160 

20 

14:09:00 

14:22:00 

14:55:00 

SSE 

16,24 

160 

10T050 

21 

06:04:00 

07:16:00 

07:42:00 

SSW 

El  3  (3  points) 

50 

11T080 

21 

08:15:00 

08:30:00 

09:1500 

SSE 

23,24 

80 

12T120 

21 

09:28:30 

09:34:00 

10:28:00 

SSE 

18,19,21,22,23,24 

120 

13T160 

21 

10:56:00 

11:15:30 

11:38:00 

SSE 

17,18,19,20,21,22,23,24 

160 

aNNW  -  North  Northwest;  SSE  -  South  Southeast;  SSW  -  South  Southwest. 
hTriethyl  phosphate. 
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Table  A2.  RTVS08  Dissemination  Log  Data 


Trial 

Name 

Date 

(June 

2008) 

Referee 

Start 

Time 

Release 

Time 

Trial 

End 

Time 

Dissemination 

Location3 

Detonation 

Points'3 

Simulant  Amount 
(kg) 

Simulant 

Typec 

Comment3 

Planned 

Actual 

01TE030 

9 

0033 

00:46:30 

01:15:30 

S2 

A 

30 

30 

TEP 

None 

02TE030 

9 

0143 

01:53:00 

02:14:00 

S2 

A 

30 

30 

TEP 

None 

03TE075 

9 

0404 

04:31 :00 

05:03:00 

S2 

B 

75 

75 

TEP 

None 

04TE075 

10 

0437 

04:47:00 

05:45:30 

S3 

B 

75 

75 

TEP 

None 

05TE150 

11 

2243 

22:53:00 

23:58:00 

N2 

B 

150 

150 

TEP 

None 

06TE030 

9 

2316 

23:46:15 

00:25:00 

S3 

A 

30 

30 

TEP 

None 

07TE030 

10 

0546 

05:58:00 

06:32:30 

S2 

A 

30 

30 

TEP 

None 

08TE075 

10 

0108 

01:23:00 

02:09:00 

S2 

B 

75 

75 

TEP 

None 

09TE150 

10 

0241 

02:46:00 

04:34:00 

S2 

B 

150 

150 

TEP 

None 

10TE030 

12 

2253 

22:55:00 

23:41:00 

WC 

NA 

30 

55.13 

TEP 

e 

11TE075 

13 

0048 

00:49:45 

01:37:00 

WC 

NA 

75 

78.16 

TEP 

D 

12TE075 

13 

0137 

01 :47:00 

02:57:00 

WC 

NA 

75 

73.77 

TEP 

E 

13TE150 

13 

0257 

03:07:00 

04:14:00 

WC 

NA 

150 

146.21 

TEP 

F 

14TE075 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

G 

15TE150 

12 

0233 

03:35:00 

05:55:30 

S2 

B 

150 

150 

TEP 

None 

16TE150 

13 

0508 

05:13:00 

07:00:00 

S2 

B 

150 

150 

TEP 

None 

17TE150 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

18MS030 

14 

0620 

06:30:00 

06:56:00 

S2 

A 

30 

30 

MeS 

None 

19M5030 

16 

0138 

01:43:00 

02:07:30 

S2 

A 

30 

30 

MeS 

None 

20BL000 

14 

0032 

00:42:00 

00:57:00 

S2 

B 

None 

None 

None 

H 

21MT030 

14 

0124 

01:40:00 

02:46:00 

S2 

A 

30/30 

30/30 

MeS/TEP 

None 

22MT075 

14 

0246 

03:17:00 

03:43:00 

S2 

B 

75 

75 

MeS/TEP 

None 

23MS030 

16 

0354 

03:55:00 

05:03:00 

S2 

A 

30 

30 

MeS 

None 

24MS075 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

25MS075 

17 

0047 

00:52:00 

01:20:00 

SI 

B 

75 

75 

MeS 

None 

26MS150 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

27MS075 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

28MS075 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

29MS150 

18 

0103 

01:11:00 

01:50:00 

N3 

B 

150 

150 

MeS 

None 

30MS150 

18 

0351 

03:52:00 

04:51:00 

N2 

B 

150 

150 

MeS 

None 
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Trial 

Date 

(June 

Referee 

Start 

Release 

Trial 

End 

Dissemination 

Detonation 

Simulant  Amount 
(kg) 

Simulant 

Comment3 

Name 

2008) 

Time 

Time 

Time 

Location3 

Points” 

Planned 

Actual 

Typec 

31MS075 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

32MS150 

18 

0442 

04:55:00 

05:17:00 

N2 

B 

150 

150 

MeS 

None 

33MS150 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

34MS150 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

35AA030 

18 

2211 

22:22:00 

22:59:00 

N3 

A 

30 

30 

AA 

None 

36AA075 

18 

2344 

23:55:00 

00:32:00 

N3 

B 

75 

75 

AA 

None 

37AA075 

19 

0112 

01:20:00 

01:44:00 

N2 

B 

75 

75 

AA 

None 

38AA150 

19 

0306 

03:14:30 

04:25:00 

N2 

B 

150 

150 

AA 

None 

39AA150 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

ND 

NA 

aS2  -  south  second;  S3  -  south  third;  WC  -  west  and  center;  ND  -  no  data;  SI  -  south  first;  N3  -  north  third;  N2  -  north  second. 
bA  -  2,  3,  6,  7,  10,  and  1 1;  B  -  1  through  12;  NA  -  not  available. 

‘TEP  -  triethyl  phosphate;  MeS  -  methyl  salicylate;  AA  -  acetic  acid. 

dC  -  40  kg  west  stack  over  a  10  min  period,  15.13  center  stack  over  a  13  min  period;  D  -  37.59  kg  west  stack  over  a  15  min  period, 
40.57  kg  center  stack  over  a  15  min  period;  E  -  38.12  kg  west  stack  over  a  15  min  period,  35.15  kg  center  stack  over  a  15  min  period; 
F  -71.48  kg  west  stack  over  a  16  min  period,  74.72  kg  center  stack  over  a  16  min  period;  G  -  trial  canceled  by  customer; 

H  -  no  simulants  used;  only  detonation  to  create  dust  cloud. 
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